2024-04-26T22:20:21.800 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T22:20:21.801 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/snapshot}} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262 --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262/orig.config.yaml 2024-04-26T22:20:21.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T22:20:21.933 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262 branch: wip-rishabh-testing-20240426.111959 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7675262' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObeXS9mii0wDMCFiv+X9R2zlzi8Rs8AhhIuJ1ApD/9aLbUfFWoc/YYibC2pW1/Rk8RhkHtYtO0U2dOUVWrsKRw= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjdXFT3YZ+ISz+3bGJFTCuOwtbluhNFlDd4BHFxuurJOc0kGD0IqPDSyoqN411sdH/p1cA71q6wJ0db5xAWskI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T22:20:21.933 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T22:20:21.933 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T22:20:21.933 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T22:20:21.935 INFO:teuthology.task.internal:Checking packages... 2024-04-26T22:20:21.958 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T22:20:21.958 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T22:20:21.958 INFO:teuthology.packaging:ref: None 2024-04-26T22:20:21.958 INFO:teuthology.packaging:tag: None 2024-04-26T22:20:21.958 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T22:20:21.958 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:20:21.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T22:20:22.101 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T22:20:22.102 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T22:20:22.109 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T22:20:22.109 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T22:20:22.130 INFO:teuthology.task.internal:Saving configuration 2024-04-26T22:20:22.146 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T22:20:22.152 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T22:20:22.176 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 22:15:14.157976', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObeXS9mii0wDMCFiv+X9R2zlzi8Rs8AhhIuJ1ApD/9aLbUfFWoc/YYibC2pW1/Rk8RhkHtYtO0U2dOUVWrsKRw='} 2024-04-26T22:20:22.201 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 22:15:14.156550', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjdXFT3YZ+ISz+3bGJFTCuOwtbluhNFlDd4BHFxuurJOc0kGD0IqPDSyoqN411sdH/p1cA71q6wJ0db5xAWskI='} 2024-04-26T22:20:22.201 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T22:20:22.207 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T22:20:22.208 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T22:20:22.208 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T22:20:22.272 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f07333eb160>, signals=[15]) 2024-04-26T22:20:22.273 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T22:20:22.279 INFO:teuthology.task.internal:Opening connections... 2024-04-26T22:20:22.279 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:20:22.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:20:22.354 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:20:22.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:20:22.428 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T22:20:22.434 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-26T22:20:22.453 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-26T22:20:22.453 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-26T22:20:22.511 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T22:20:22.512 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T22:20:22.512 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-26T22:20:22.540 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-04-26T22:20:22.558 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-04-26T22:20:22.558 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-26T22:20:22.615 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T22:20:22.616 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T22:20:22.616 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-26T22:20:22.665 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T22:20:22.673 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T22:20:22.679 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T22:20:22.679 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-26T22:20:22.682 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-04-26T22:20:22.698 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T22:20:22.703 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T22:20:22.704 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-26T22:20:22.739 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-04-26T22:20:22.849 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T22:20:22.908 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T22:20:22.921 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T22:20:22.921 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T22:20:22.921 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T22:20:22.921 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T22:20:22.922 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T22:20:22.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:20:22.938 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-04-26T22:20:22.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:20:22.939 DEBUG:teuthology.orchestra.run.smithi062:> uname -r 2024-04-26T22:20:22.995 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-437.el9.x86_64 2024-04-26T22:20:22.995 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-437.el9.x86_64 2024-04-26T22:20:22.995 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2024-04-26T22:20:22.998 INFO:teuthology.orchestra.run.smithi062.stdout:5.14.0-437.el9.x86_64 2024-04-26T22:20:22.998 INFO:teuthology.task.kernel:Running kernel on smithi062: 5.14.0-437.el9.x86_64 2024-04-26T22:20:22.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum install -y kernel 2024-04-26T22:20:25.291 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T22:20:25.399 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T22:20:27.195 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T22:20:27.550 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-26T22:20:32.450 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-04-26T22:20:32.551 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-04-26T22:20:34.351 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 16 kB 00:00 2024-04-26T22:20:34.384 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-26T22:20:36.645 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 21 MB 00:02 2024-04-26T22:20:36.863 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-04-26T22:20:41.719 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T22:20:42.069 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-04-26T22:20:43.600 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T22:20:43.600 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T22:20:43.690 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-04-26T22:20:43.691 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-04-26T22:20:43.691 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-04-26T22:20:43.799 DEBUG:teuthology.orchestra.run.smithi174:> echo no | sudo yum reinstall kernel || true 2024-04-26T22:20:43.983 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T22:20:43.983 INFO:teuthology.orchestra.run.smithi062.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T22:20:44.077 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-26T22:20:44.079 INFO:teuthology.orchestra.run.smithi062.stdout:Nothing to do. 2024-04-26T22:20:44.079 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-26T22:20:44.207 DEBUG:teuthology.orchestra.run.smithi062:> echo no | sudo yum reinstall kernel || true 2024-04-26T22:20:44.295 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 10:20:41 PM UTC. 2024-04-26T22:20:44.421 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-04-26T22:20:44.432 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:44.432 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-04-26T22:20:44.432 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:44.432 INFO:teuthology.orchestra.run.smithi174.stdout:Reinstalling: 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 255 k 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 0 2024-04-26T22:20:44.433 INFO:teuthology.orchestra.run.smithi174.stderr:Operation aborted. 2024-04-26T22:20:44.491 INFO:teuthology.orchestra.run.smithi174.stdout:Is this ok [y/N]: 2024-04-26T22:20:44.491 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum reinstall -y kernel || true 2024-04-26T22:20:44.730 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 10:20:42 PM UTC. 2024-04-26T22:20:44.858 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-26T22:20:44.880 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalling: 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:44.881 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:44.882 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 255 k 2024-04-26T22:20:44.882 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 0 2024-04-26T22:20:44.882 INFO:teuthology.orchestra.run.smithi062.stderr:Operation aborted. 2024-04-26T22:20:44.941 INFO:teuthology.orchestra.run.smithi062.stdout:Is this ok [y/N]: 2024-04-26T22:20:44.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum reinstall -y kernel || true 2024-04-26T22:20:44.984 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 10:20:41 PM UTC. 2024-04-26T22:20:45.100 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout:Reinstalling: 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:45.101 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-04-26T22:20:45.102 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-04-26T22:20:45.102 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:45.102 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 255 k 2024-04-26T22:20:45.102 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 0 2024-04-26T22:20:45.102 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-04-26T22:20:45.456 INFO:teuthology.orchestra.run.smithi062.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 10:20:42 PM UTC. 2024-04-26T22:20:45.576 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout: Package Architecture Version Repository Size 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalling: 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:45.577 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-26T22:20:45.578 INFO:teuthology.orchestra.run.smithi062.stdout:================================================================================ 2024-04-26T22:20:45.578 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:45.578 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 255 k 2024-04-26T22:20:45.578 INFO:teuthology.orchestra.run.smithi062.stdout:Installed size: 0 2024-04-26T22:20:45.578 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-26T22:20:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-437.el9.x86_64.rpm 548 kB/s | 255 kB 00:00 2024-04-26T22:20:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:20:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:Total 346 kB/s | 255 kB 00:00 2024-04-26T22:20:45.854 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-04-26T22:20:45.862 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-26T22:20:45.862 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:20:45.862 INFO:teuthology.orchestra.run.smithi062.stdout:Total 895 kB/s | 255 kB 00:00 2024-04-26T22:20:45.883 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-26T22:20:46.049 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-04-26T22:20:46.049 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-04-26T22:20:46.054 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-04-26T22:20:46.055 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-04-26T22:20:46.072 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-26T22:20:46.072 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-26T22:20:46.077 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-26T22:20:46.077 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-26T22:20:46.182 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-04-26T22:20:46.215 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-26T22:20:46.255 INFO:teuthology.orchestra.run.smithi174.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T22:20:46.255 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:46.288 INFO:teuthology.orchestra.run.smithi062.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T22:20:46.288 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:46.973 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:46.973 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T22:20:47.009 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:47.009 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T22:20:47.629 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:47.629 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:47.629 INFO:teuthology.orchestra.run.smithi174.stdout:Reinstalled: 2024-04-26T22:20:47.630 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T22:20:47.630 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:20:47.630 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout:Reinstalled: 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:20:47.731 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-26T22:20:47.836 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T22:20:47.882 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T22:20:47.882 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T22:20:47.883 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T22:20:47.883 DEBUG:teuthology.parallel:result is None 2024-04-26T22:20:47.910 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T22:20:47.997 INFO:teuthology.orchestra.run.smithi062.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T22:20:47.998 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T22:20:47.998 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T22:20:47.999 DEBUG:teuthology.parallel:result is None 2024-04-26T22:20:47.999 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T22:20:48.006 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T22:20:48.007 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T22:20:48.009 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T22:20:48.023 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T22:20:48.029 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T22:20:48.039 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T22:20:48.040 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T22:20:48.064 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T22:20:48.094 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T22:20:48.101 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T22:20:48.101 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T22:20:48.125 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T22:20:48.155 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:20:48.172 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:20:48.183 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:20:48.202 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T22:20:48.204 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T22:20:48.211 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T22:20:48.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T22:20:48.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T22:20:48.239 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T22:20:48.247 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T22:20:48.247 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T22:20:48.280 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T22:20:48.294 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:20:48.371 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:20:48.422 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:20:48.504 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:20:48.543 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:20:48.543 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T22:20:48.607 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:20:48.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T22:20:48.696 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:20:48.780 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T22:20:48.816 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:20:48.816 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T22:20:48.878 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-26T22:20:48.881 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-04-26T22:20:48.915 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:20:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T22:20:49.460 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T22:20:49.473 INFO:teuthology.task.internal:Starting timer... 2024-04-26T22:20:49.473 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T22:20:49.484 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T22:20:49.492 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T22:20:49.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-26T22:20:49.563 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-26T22:20:49.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-04-26T22:20:49.603 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-04-26T22:20:49.605 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T22:20:49.605 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-26T22:20:49.631 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-26T22:20:49.631 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-04-26T22:20:49.667 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-04-26T22:20:49.667 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-04-26T22:20:49.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:20:49.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:20:49.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T22:20:49.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:20:49.731 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T22:20:49.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-26T22:20:49.767 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-04-26T22:20:49.794 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T22:20:49.803 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T22:20:49.808 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T22:20:50.005 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T22:20:50.028 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T22:20:50.029 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi062.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T22:24:20.826 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-04-26T22:24:20.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-26T22:24:20.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:24:20.902 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-26T22:24:20.981 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-26T22:24:20.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-26T22:24:20.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T22:24:21.057 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-04-26T22:24:21.145 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-26T22:24:21.146 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T22:24:21.156 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T22:24:21.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T22:24:21.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:24:21.159 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T22:24:21.159 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T22:24:21.186 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T22:24:21.200 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T22:24:21.212 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T22:24:21.224 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-26T22:24:21.225 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T22:24:21.234 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-26T22:24:21.248 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T22:24:21.248 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-04-26T22:24:21.258 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-04-26T22:24:21.261 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T22:24:21.271 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T22:24:21.283 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T22:24:21.319 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.343 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-04-26T22:24:21.347 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T22:24:21.347 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-04-26T22:24:21.347 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.347 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.347 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.348 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T22:24:21.348 INFO:teuthology.run_tasks:Running task install... 2024-04-26T22:24:21.356 DEBUG:teuthology.task.install:project ceph 2024-04-26T22:24:21.356 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T22:24:21.356 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T22:24:21.356 INFO:teuthology.task.install:Using flavor: default 2024-04-26T22:24:21.362 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T22:24:21.362 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T22:24:21.362 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T22:24:21.363 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:24:21.365 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T22:24:21.365 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:24:21.581 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T22:24:21.581 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T22:24:21.585 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T22:24:21.585 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T22:24:21.692 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T22:24:21.693 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:24:21.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T22:24:21.719 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T22:24:21.719 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T22:24:21.754 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T22:24:21.755 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:24:21.755 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T22:24:21.780 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T22:24:21.780 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T22:24:21.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T22:24:21.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T22:24:21.858 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T22:24:21.917 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T22:24:21.921 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-26T22:24:21.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-26T22:24:21.981 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-04-26T22:24:21.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-04-26T22:24:22.156 INFO:teuthology.orchestra.run.smithi062.stdout:48 files removed 2024-04-26T22:24:22.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T22:24:22.206 INFO:teuthology.orchestra.run.smithi174.stdout:48 files removed 2024-04-26T22:24:22.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T22:24:22.922 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 179 kB/s | 84 kB 00:00 2024-04-26T22:24:22.949 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 183 kB/s | 84 kB 00:00 2024-04-26T22:24:23.319 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-04-26T22:24:23.348 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T22:24:23.710 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-26T22:24:23.723 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-26T22:24:24.282 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-26T22:24:25.185 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.0 MB 00:01 2024-04-26T22:24:26.041 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-26T22:24:31.401 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-26T22:24:33.277 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-26T22:24:35.306 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-26T22:24:40.662 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-26T22:24:42.922 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:24:42.922 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:24:42.943 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T22:24:42.943 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T22:24:42.944 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T22:24:42.945 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T22:24:42.947 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T22:24:42.947 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T22:24:42.947 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T22:24:43.005 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-26T22:24:43.014 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-26T22:24:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-26T22:24:43.014 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-26T22:24:43.014 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-26T22:24:43.014 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T22:24:43.015 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T22:24:43.016 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T22:24:43.017 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:24:43.018 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T22:24:43.019 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T22:24:43.020 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T22:24:43.021 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T22:24:43.022 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T22:24:43.023 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T22:24:43.024 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T22:24:43.025 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T22:24:43.026 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T22:24:43.027 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T22:24:43.028 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T22:24:43.029 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T22:24:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T22:24:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T22:24:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T22:24:43.030 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-26T22:24:43.031 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-04-26T22:24:43.032 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 4 Packages 2024-04-26T22:24:43.032 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:24:43.033 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 222 M 2024-04-26T22:24:43.033 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-26T22:24:43.913 INFO:teuthology.orchestra.run.smithi062.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-26T22:24:44.155 INFO:teuthology.orchestra.run.smithi062.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 3.6 MB/s | 889 kB 00:00 2024-04-26T22:24:44.205 INFO:teuthology.orchestra.run.smithi062.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.8 MB/s | 145 kB 00:00 2024-04-26T22:24:44.555 INFO:teuthology.orchestra.run.smithi062.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 6.7 MB/s | 2.4 MB 00:00 2024-04-26T22:24:44.583 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-26T22:24:44.647 INFO:teuthology.orchestra.run.smithi062.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-04-26T22:24:44.831 INFO:teuthology.orchestra.run.smithi062.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-04-26T22:24:45.197 INFO:teuthology.orchestra.run.smithi062.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.5 MB/s | 4.7 MB 00:00 2024-04-26T22:24:46.098 INFO:teuthology.orchestra.run.smithi062.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 9.0 MB/s | 21 MB 00:02 2024-04-26T22:24:46.148 INFO:teuthology.orchestra.run.smithi062.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 506 kB/s | 25 kB 00:00 2024-04-26T22:24:46.348 INFO:teuthology.orchestra.run.smithi062.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 9.2 MB/s | 11 MB 00:01 2024-04-26T22:24:46.431 INFO:teuthology.orchestra.run.smithi062.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.6 MB/s | 225 kB 00:00 2024-04-26T22:24:46.515 INFO:teuthology.orchestra.run.smithi062.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 324 kB/s | 27 kB 00:00 2024-04-26T22:24:46.591 INFO:teuthology.orchestra.run.smithi062.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 423 kB/s | 32 kB 00:00 2024-04-26T22:24:46.749 INFO:teuthology.orchestra.run.smithi062.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.6 MB/s | 747 kB 00:00 2024-04-26T22:24:46.824 INFO:teuthology.orchestra.run.smithi062.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 163 kB 00:00 2024-04-26T22:24:46.900 INFO:teuthology.orchestra.run.smithi062.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T22:24:47.067 INFO:teuthology.orchestra.run.smithi062.stdout:(17/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 7.5 MB/s | 17 MB 00:02 2024-04-26T22:24:47.100 INFO:teuthology.orchestra.run.smithi062.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 2.4 MB/s | 501 kB 00:00 2024-04-26T22:24:47.159 INFO:teuthology.orchestra.run.smithi062.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 773 kB/s | 45 kB 00:00 2024-04-26T22:24:47.226 INFO:teuthology.orchestra.run.smithi062.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T22:24:47.301 INFO:teuthology.orchestra.run.smithi062.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T22:24:47.385 INFO:teuthology.orchestra.run.smithi062.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T22:24:47.469 INFO:teuthology.orchestra.run.smithi062.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T22:24:47.544 INFO:teuthology.orchestra.run.smithi062.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.3 MB/s | 99 kB 00:00 2024-04-26T22:24:47.619 INFO:teuthology.orchestra.run.smithi062.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-26T22:24:47.703 INFO:teuthology.orchestra.run.smithi062.stdout:(26/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 8.4 MB/s | 5.4 MB 00:00 2024-04-26T22:24:47.787 INFO:teuthology.orchestra.run.smithi062.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T22:24:47.862 INFO:teuthology.orchestra.run.smithi062.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 457 kB/s | 34 kB 00:00 2024-04-26T22:24:47.945 INFO:teuthology.orchestra.run.smithi062.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.7 MB/s | 142 kB 00:00 2024-04-26T22:24:48.004 INFO:teuthology.orchestra.run.smithi062.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.0 MB/s | 3.1 MB 00:00 2024-04-26T22:24:48.454 INFO:teuthology.orchestra.run.smithi062.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.0 MB/s | 3.5 MB 00:00 2024-04-26T22:24:48.538 INFO:teuthology.orchestra.run.smithi062.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.9 MB/s | 249 kB 00:00 2024-04-26T22:24:48.621 INFO:teuthology.orchestra.run.smithi062.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 592 kB/s | 49 kB 00:00 2024-04-26T22:24:48.705 INFO:teuthology.orchestra.run.smithi062.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 198 kB/s | 16 kB 00:00 2024-04-26T22:24:48.838 INFO:teuthology.orchestra.run.smithi062.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-26T22:24:48.864 INFO:teuthology.orchestra.run.smithi062.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 273 kB 00:00 2024-04-26T22:24:48.939 INFO:teuthology.orchestra.run.smithi062.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 243 kB/s | 18 kB 00:00 2024-04-26T22:24:48.964 INFO:teuthology.orchestra.run.smithi062.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 5.9 MB/s | 762 kB 00:00 2024-04-26T22:24:49.065 INFO:teuthology.orchestra.run.smithi062.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 642 kB/s | 80 kB 00:00 2024-04-26T22:24:49.142 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 6.2 MB 00:00 2024-04-26T22:24:49.490 INFO:teuthology.orchestra.run.smithi062.stdout:(40/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 15 MB/s | 49 MB 00:03 2024-04-26T22:24:49.565 INFO:teuthology.orchestra.run.smithi062.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-04-26T22:24:49.607 INFO:teuthology.orchestra.run.smithi062.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 360 kB/s | 195 kB 00:00 2024-04-26T22:24:49.641 INFO:teuthology.orchestra.run.smithi062.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-26T22:24:49.708 INFO:teuthology.orchestra.run.smithi062.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T22:24:49.724 INFO:teuthology.orchestra.run.smithi062.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T22:24:49.791 INFO:teuthology.orchestra.run.smithi062.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-26T22:24:49.825 INFO:teuthology.orchestra.run.smithi062.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T22:24:49.917 INFO:teuthology.orchestra.run.smithi062.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-04-26T22:24:50.117 INFO:teuthology.orchestra.run.smithi062.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-26T22:24:50.184 INFO:teuthology.orchestra.run.smithi062.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-26T22:24:50.209 INFO:teuthology.orchestra.run.smithi062.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 407 kB/s | 107 kB 00:00 2024-04-26T22:24:50.260 INFO:teuthology.orchestra.run.smithi062.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-04-26T22:24:50.335 INFO:teuthology.orchestra.run.smithi062.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-26T22:24:50.360 INFO:teuthology.orchestra.run.smithi062.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 217 kB/s | 33 kB 00:00 2024-04-26T22:24:50.461 INFO:teuthology.orchestra.run.smithi062.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 145 kB/s | 18 kB 00:00 2024-04-26T22:24:50.486 INFO:teuthology.orchestra.run.smithi062.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-26T22:24:50.578 INFO:teuthology.orchestra.run.smithi062.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-04-26T22:24:50.603 INFO:teuthology.orchestra.run.smithi062.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-04-26T22:24:50.628 INFO:teuthology.orchestra.run.smithi062.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 958 kB/s | 160 kB 00:00 2024-04-26T22:24:50.662 INFO:teuthology.orchestra.run.smithi062.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-26T22:24:50.676 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-26T22:24:50.695 INFO:teuthology.orchestra.run.smithi062.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-04-26T22:24:50.720 INFO:teuthology.orchestra.run.smithi062.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-04-26T22:24:50.746 INFO:teuthology.orchestra.run.smithi062.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-26T22:24:50.771 INFO:teuthology.orchestra.run.smithi062.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-26T22:24:50.796 INFO:teuthology.orchestra.run.smithi062.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-26T22:24:50.822 INFO:teuthology.orchestra.run.smithi062.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-04-26T22:24:50.989 INFO:teuthology.orchestra.run.smithi062.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-26T22:24:51.157 INFO:teuthology.orchestra.run.smithi062.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-26T22:24:51.231 INFO:teuthology.orchestra.run.smithi062.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-26T22:24:51.264 INFO:teuthology.orchestra.run.smithi062.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-26T22:24:51.356 INFO:teuthology.orchestra.run.smithi062.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T22:24:51.407 INFO:teuthology.orchestra.run.smithi062.stdout:(72/152): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-04-26T22:24:51.459 INFO:teuthology.orchestra.run.smithi062.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 856 kB/s | 87 kB 00:00 2024-04-26T22:24:51.559 INFO:teuthology.orchestra.run.smithi062.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-26T22:24:51.584 INFO:teuthology.orchestra.run.smithi062.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-26T22:24:51.635 INFO:teuthology.orchestra.run.smithi062.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-26T22:24:51.652 INFO:teuthology.orchestra.run.smithi062.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-04-26T22:24:51.685 INFO:teuthology.orchestra.run.smithi062.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-26T22:24:51.719 INFO:teuthology.orchestra.run.smithi062.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-26T22:24:51.769 INFO:teuthology.orchestra.run.smithi062.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 415 kB/s | 35 kB 00:00 2024-04-26T22:24:51.844 INFO:teuthology.orchestra.run.smithi062.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T22:24:51.911 INFO:teuthology.orchestra.run.smithi062.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T22:24:51.986 INFO:teuthology.orchestra.run.smithi062.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-26T22:24:52.028 INFO:teuthology.orchestra.run.smithi062.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-04-26T22:24:52.054 INFO:teuthology.orchestra.run.smithi062.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-04-26T22:24:52.104 INFO:teuthology.orchestra.run.smithi062.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T22:24:52.129 INFO:teuthology.orchestra.run.smithi062.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T22:24:52.205 INFO:teuthology.orchestra.run.smithi062.stdout:(88/152): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-04-26T22:24:52.230 INFO:teuthology.orchestra.run.smithi062.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-26T22:24:52.255 INFO:teuthology.orchestra.run.smithi062.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 406 kB/s | 51 kB 00:00 2024-04-26T22:24:52.280 INFO:teuthology.orchestra.run.smithi062.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-04-26T22:24:52.339 INFO:teuthology.orchestra.run.smithi062.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 500 kB/s | 42 kB 00:00 2024-04-26T22:24:52.364 INFO:teuthology.orchestra.run.smithi062.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 530 kB/s | 44 kB 00:00 2024-04-26T22:24:52.431 INFO:teuthology.orchestra.run.smithi062.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-04-26T22:24:52.464 INFO:teuthology.orchestra.run.smithi062.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-04-26T22:24:52.531 INFO:teuthology.orchestra.run.smithi062.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-26T22:24:52.573 INFO:teuthology.orchestra.run.smithi062.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-04-26T22:24:52.665 INFO:teuthology.orchestra.run.smithi062.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 432 kB/s | 58 kB 00:00 2024-04-26T22:24:52.699 INFO:teuthology.orchestra.run.smithi062.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 749 kB/s | 94 kB 00:00 2024-04-26T22:24:52.807 INFO:teuthology.orchestra.run.smithi062.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 448 kB/s | 64 kB 00:00 2024-04-26T22:24:52.841 INFO:teuthology.orchestra.run.smithi062.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 315 kB/s | 45 kB 00:00 2024-04-26T22:24:53.008 INFO:teuthology.orchestra.run.smithi062.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-26T22:24:53.475 INFO:teuthology.orchestra.run.smithi062.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-26T22:24:53.500 INFO:teuthology.orchestra.run.smithi062.stdout:(104/152): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-04-26T22:24:53.567 INFO:teuthology.orchestra.run.smithi062.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-04-26T22:24:53.593 INFO:teuthology.orchestra.run.smithi062.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-26T22:24:53.784 INFO:teuthology.orchestra.run.smithi062.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 136 kB/s | 26 kB 00:00 2024-04-26T22:24:53.843 INFO:teuthology.orchestra.run.smithi062.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-26T22:24:53.868 INFO:teuthology.orchestra.run.smithi062.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 587 kB/s | 49 kB 00:00 2024-04-26T22:24:53.910 INFO:teuthology.orchestra.run.smithi062.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-26T22:24:53.944 INFO:teuthology.orchestra.run.smithi062.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T22:24:53.952 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-26T22:24:54.002 INFO:teuthology.orchestra.run.smithi062.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-04-26T22:24:54.028 INFO:teuthology.orchestra.run.smithi062.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-26T22:24:54.070 INFO:teuthology.orchestra.run.smithi062.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 658 kB/s | 43 kB 00:00 2024-04-26T22:24:54.095 INFO:teuthology.orchestra.run.smithi062.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-26T22:24:54.137 INFO:teuthology.orchestra.run.smithi062.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-26T22:24:54.170 INFO:teuthology.orchestra.run.smithi062.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-26T22:24:54.234 INFO:teuthology.orchestra.run.smithi062.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.6 MB/s | 358 kB 00:00 2024-04-26T22:24:54.262 INFO:teuthology.orchestra.run.smithi062.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T22:24:54.304 INFO:teuthology.orchestra.run.smithi062.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 505 kB/s | 35 kB 00:00 2024-04-26T22:24:54.330 INFO:teuthology.orchestra.run.smithi062.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T22:24:54.413 INFO:teuthology.orchestra.run.smithi062.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-26T22:24:54.439 INFO:teuthology.orchestra.run.smithi062.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-26T22:24:54.472 INFO:teuthology.orchestra.run.smithi062.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-26T22:24:54.514 INFO:teuthology.orchestra.run.smithi062.stdout:(125/152): protobuf-compiler-3.14.0-13.el9.x86_ 516 kB/s | 863 kB 00:01 2024-04-26T22:24:54.531 INFO:teuthology.orchestra.run.smithi062.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-26T22:24:54.556 INFO:teuthology.orchestra.run.smithi062.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-26T22:24:54.598 INFO:teuthology.orchestra.run.smithi062.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-04-26T22:24:54.623 INFO:teuthology.orchestra.run.smithi062.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-26T22:24:54.657 INFO:teuthology.orchestra.run.smithi062.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T22:24:54.682 INFO:teuthology.orchestra.run.smithi062.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-26T22:24:54.699 INFO:teuthology.orchestra.run.smithi062.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-26T22:24:54.725 INFO:teuthology.orchestra.run.smithi062.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-04-26T22:24:54.758 INFO:teuthology.orchestra.run.smithi062.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-26T22:24:54.800 INFO:teuthology.orchestra.run.smithi062.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-26T22:24:54.834 INFO:teuthology.orchestra.run.smithi062.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 824 kB/s | 90 kB 00:00 2024-04-26T22:24:54.859 INFO:teuthology.orchestra.run.smithi062.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 280 kB/s | 28 kB 00:00 2024-04-26T22:24:54.884 INFO:teuthology.orchestra.run.smithi062.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-26T22:24:54.910 INFO:teuthology.orchestra.run.smithi062.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T22:24:54.935 INFO:teuthology.orchestra.run.smithi062.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-04-26T22:24:54.960 INFO:teuthology.orchestra.run.smithi062.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-04-26T22:24:54.985 INFO:teuthology.orchestra.run.smithi062.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-04-26T22:24:55.011 INFO:teuthology.orchestra.run.smithi062.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T22:24:55.036 INFO:teuthology.orchestra.run.smithi062.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T22:24:55.061 INFO:teuthology.orchestra.run.smithi062.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-26T22:24:55.086 INFO:teuthology.orchestra.run.smithi062.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-26T22:24:55.118 INFO:teuthology.orchestra.run.smithi062.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-04-26T22:24:55.162 INFO:teuthology.orchestra.run.smithi062.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-26T22:24:55.445 INFO:teuthology.orchestra.run.smithi062.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 9.5 MB/s | 3.4 MB 00:00 2024-04-26T22:24:55.487 INFO:teuthology.orchestra.run.smithi062.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 1.3 MB 00:00 2024-04-26T22:24:55.538 INFO:teuthology.orchestra.run.smithi062.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 7.5 MB/s | 3.1 MB 00:00 2024-04-26T22:24:55.971 INFO:teuthology.orchestra.run.smithi062.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 2.4 MB 00:00 2024-04-26T22:24:55.978 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:24:55.978 INFO:teuthology.orchestra.run.smithi062.stdout:Total 17 MB/s | 222 MB 00:12 2024-04-26T22:24:57.074 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-26T22:24:57.233 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-26T22:24:57.233 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-26T22:24:58.952 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T22:24:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-26T22:24:59.123 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-26T22:25:01.064 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:25:01.065 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T22:25:01.075 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T22:25:01.076 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T22:25:01.077 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T22:25:01.078 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T22:25:01.080 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T22:25:01.082 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T22:25:01.083 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T22:25:01.144 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T22:25:01.153 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T22:25:01.154 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T22:25:01.155 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T22:25:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T22:25:01.157 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T22:25:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T22:25:01.159 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T22:25:01.160 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T22:25:01.161 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T22:25:01.162 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T22:25:01.163 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T22:25:01.164 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 4 Packages 2024-04-26T22:25:01.165 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:01.166 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 222 M 2024-04-26T22:25:01.166 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-04-26T22:25:02.289 INFO:teuthology.orchestra.run.smithi174.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-26T22:25:03.298 INFO:teuthology.orchestra.run.smithi174.stdout:(2/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 17 MB/s | 21 MB 00:01 2024-04-26T22:25:03.332 INFO:teuthology.orchestra.run.smithi174.stdout:(3/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 854 kB/s | 889 kB 00:01 2024-04-26T22:25:03.356 INFO:teuthology.orchestra.run.smithi174.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 2.4 MB/s | 145 kB 00:00 2024-04-26T22:25:03.457 INFO:teuthology.orchestra.run.smithi174.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 15 MB/s | 1.5 MB 00:00 2024-04-26T22:25:03.732 INFO:teuthology.orchestra.run.smithi174.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 17 MB/s | 4.7 MB 00:00 2024-04-26T22:25:04.240 INFO:teuthology.orchestra.run.smithi174.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-26T22:25:04.590 INFO:teuthology.orchestra.run.smithi174.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 20 MB/s | 17 MB 00:00 2024-04-26T22:25:04.641 INFO:teuthology.orchestra.run.smithi174.stdout:(9/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-26T22:25:04.666 INFO:teuthology.orchestra.run.smithi174.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 335 kB/s | 25 kB 00:00 2024-04-26T22:25:04.716 INFO:teuthology.orchestra.run.smithi174.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 4.4 MB/s | 225 kB 00:00 2024-04-26T22:25:04.766 INFO:teuthology.orchestra.run.smithi174.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 542 kB/s | 27 kB 00:00 2024-04-26T22:25:04.816 INFO:teuthology.orchestra.run.smithi174.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 636 kB/s | 32 kB 00:00 2024-04-26T22:25:04.892 INFO:teuthology.orchestra.run.smithi174.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 9.7 MB/s | 747 kB 00:00 2024-04-26T22:25:04.950 INFO:teuthology.orchestra.run.smithi174.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 163 kB 00:00 2024-04-26T22:25:05.017 INFO:teuthology.orchestra.run.smithi174.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.9 MB/s | 127 kB 00:00 2024-04-26T22:25:05.101 INFO:teuthology.orchestra.run.smithi174.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.9 MB/s | 501 kB 00:00 2024-04-26T22:25:05.317 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-26T22:25:05.408 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T22:25:05.434 INFO:teuthology.orchestra.run.smithi174.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 16 MB/s | 5.4 MB 00:00 2024-04-26T22:25:05.447 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T22:25:05.485 INFO:teuthology.orchestra.run.smithi174.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 909 kB/s | 45 kB 00:00 2024-04-26T22:25:05.501 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T22:25:05.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T22:25:05.543 INFO:teuthology.orchestra.run.smithi174.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.3 MB/s | 135 kB 00:00 2024-04-26T22:25:05.568 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T22:25:05.593 INFO:teuthology.orchestra.run.smithi174.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 3.2 MB/s | 161 kB 00:00 2024-04-26T22:25:05.611 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T22:25:05.660 INFO:teuthology.orchestra.run.smithi174.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 4.7 MB/s | 321 kB 00:00 2024-04-26T22:25:05.719 INFO:teuthology.orchestra.run.smithi174.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 5.0 MB/s | 298 kB 00:00 2024-04-26T22:25:05.786 INFO:teuthology.orchestra.run.smithi174.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T22:25:05.853 INFO:teuthology.orchestra.run.smithi174.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T22:25:05.956 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T22:25:06.060 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:25:06.095 INFO:teuthology.orchestra.run.smithi174.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 13 MB/s | 3.1 MB 00:00 2024-04-26T22:25:06.154 INFO:teuthology.orchestra.run.smithi174.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T22:25:06.185 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:25:06.211 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:25:06.221 INFO:teuthology.orchestra.run.smithi174.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 509 kB/s | 34 kB 00:00 2024-04-26T22:25:06.288 INFO:teuthology.orchestra.run.smithi174.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 2.1 MB/s | 142 kB 00:00 2024-04-26T22:25:06.303 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:25:06.332 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:25:06.488 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T22:25:06.538 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T22:25:06.565 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:25:06.572 INFO:teuthology.orchestra.run.smithi174.stdout:(30/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.5 MB/s | 11 MB 00:02 2024-04-26T22:25:06.630 INFO:teuthology.orchestra.run.smithi174.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 10 MB/s | 3.5 MB 00:00 2024-04-26T22:25:06.670 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:25:06.699 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:25:06.705 INFO:teuthology.orchestra.run.smithi174.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.3 MB/s | 249 kB 00:00 2024-04-26T22:25:06.747 INFO:teuthology.orchestra.run.smithi174.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 1.2 MB/s | 49 kB 00:00 2024-04-26T22:25:06.762 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:25:06.783 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:25:06.789 INFO:teuthology.orchestra.run.smithi174.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 396 kB/s | 16 kB 00:00 2024-04-26T22:25:06.808 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:25:06.856 INFO:teuthology.orchestra.run.smithi174.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 4.0 MB/s | 273 kB 00:00 2024-04-26T22:25:06.940 INFO:teuthology.orchestra.run.smithi174.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 9.0 MB/s | 762 kB 00:00 2024-04-26T22:25:06.990 INFO:teuthology.orchestra.run.smithi174.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 366 kB/s | 18 kB 00:00 2024-04-26T22:25:07.007 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:25:07.047 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:25:07.099 INFO:teuthology.orchestra.run.smithi174.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-26T22:25:07.216 INFO:teuthology.orchestra.run.smithi174.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.7 MB/s | 803 kB 00:00 2024-04-26T22:25:07.217 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T22:25:07.266 INFO:teuthology.orchestra.run.smithi174.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-26T22:25:07.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T22:25:07.316 INFO:teuthology.orchestra.run.smithi174.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-26T22:25:07.349 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T22:25:07.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T22:25:07.399 INFO:teuthology.orchestra.run.smithi174.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-26T22:25:07.442 INFO:teuthology.orchestra.run.smithi174.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-26T22:25:07.460 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T22:25:07.492 INFO:teuthology.orchestra.run.smithi174.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-26T22:25:07.500 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T22:25:07.542 INFO:teuthology.orchestra.run.smithi174.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T22:25:07.544 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T22:25:07.576 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T22:25:07.584 INFO:teuthology.orchestra.run.smithi174.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-26T22:25:07.604 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T22:25:07.675 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T22:25:07.727 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T22:25:07.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T22:25:07.768 INFO:teuthology.orchestra.run.smithi174.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-26T22:25:07.792 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T22:25:07.860 INFO:teuthology.orchestra.run.smithi174.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-04-26T22:25:07.870 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T22:25:07.902 INFO:teuthology.orchestra.run.smithi174.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-04-26T22:25:07.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T22:25:07.946 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T22:25:07.952 INFO:teuthology.orchestra.run.smithi174.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-26T22:25:07.974 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T22:25:08.019 INFO:teuthology.orchestra.run.smithi174.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-26T22:25:08.044 INFO:teuthology.orchestra.run.smithi174.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 581 kB/s | 107 kB 00:00 2024-04-26T22:25:08.048 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T22:25:08.069 INFO:teuthology.orchestra.run.smithi174.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-04-26T22:25:08.098 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T22:25:08.175 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T22:25:08.186 INFO:teuthology.orchestra.run.smithi174.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-04-26T22:25:08.253 INFO:teuthology.orchestra.run.smithi174.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-26T22:25:08.278 INFO:teuthology.orchestra.run.smithi174.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-26T22:25:08.304 INFO:teuthology.orchestra.run.smithi174.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-26T22:25:08.320 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T22:25:08.329 INFO:teuthology.orchestra.run.smithi174.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-04-26T22:25:08.367 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T22:25:08.396 INFO:teuthology.orchestra.run.smithi174.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-26T22:25:08.430 INFO:teuthology.orchestra.run.smithi174.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-26T22:25:08.455 INFO:teuthology.orchestra.run.smithi174.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-26T22:25:08.480 INFO:teuthology.orchestra.run.smithi174.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-26T22:25:08.514 INFO:teuthology.orchestra.run.smithi174.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-26T22:25:08.656 INFO:teuthology.orchestra.run.smithi174.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-04-26T22:25:08.673 INFO:teuthology.orchestra.run.smithi174.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 222 kB/s | 35 kB 00:00 2024-04-26T22:25:08.925 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T22:25:09.006 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T22:25:09.041 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T22:25:09.084 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T22:25:09.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T22:25:09.156 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T22:25:09.165 INFO:teuthology.orchestra.run.smithi174.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 9.3 MB/s | 4.7 MB 00:00 2024-04-26T22:25:09.333 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T22:25:09.392 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T22:25:09.449 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T22:25:09.522 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T22:25:09.558 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T22:25:09.582 INFO:teuthology.orchestra.run.smithi174.stdout:(67/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 9.9 MB/s | 49 MB 00:04 2024-04-26T22:25:09.595 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T22:25:09.640 INFO:teuthology.orchestra.run.smithi174.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-04-26T22:25:09.643 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T22:25:09.678 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T22:25:09.690 INFO:teuthology.orchestra.run.smithi174.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-04-26T22:25:09.713 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T22:25:09.716 INFO:teuthology.orchestra.run.smithi174.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-26T22:25:09.756 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T22:25:09.808 INFO:teuthology.orchestra.run.smithi174.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-26T22:25:09.899 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T22:25:09.900 INFO:teuthology.orchestra.run.smithi174.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-26T22:25:09.925 INFO:teuthology.orchestra.run.smithi174.stdout:(73/152): postgresql-contrib-13.14-1.el9.x86_64 2.6 MB/s | 911 kB 00:00 2024-04-26T22:25:09.943 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T22:25:09.950 INFO:teuthology.orchestra.run.smithi174.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 615 kB/s | 87 kB 00:00 2024-04-26T22:25:09.974 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T22:25:09.983 INFO:teuthology.orchestra.run.smithi174.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T22:25:10.009 INFO:teuthology.orchestra.run.smithi174.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-26T22:25:10.016 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T22:25:10.051 INFO:teuthology.orchestra.run.smithi174.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-04-26T22:25:10.101 INFO:teuthology.orchestra.run.smithi174.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-04-26T22:25:10.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T22:25:10.126 INFO:teuthology.orchestra.run.smithi174.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T22:25:10.149 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T22:25:10.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T22:25:10.201 INFO:teuthology.orchestra.run.smithi174.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-26T22:25:10.214 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T22:25:10.227 INFO:teuthology.orchestra.run.smithi174.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 277 kB/s | 35 kB 00:00 2024-04-26T22:25:10.239 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T22:25:10.317 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T22:25:10.335 INFO:teuthology.orchestra.run.smithi174.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-26T22:25:10.361 INFO:teuthology.orchestra.run.smithi174.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-26T22:25:10.386 INFO:teuthology.orchestra.run.smithi174.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T22:25:10.411 INFO:teuthology.orchestra.run.smithi174.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-26T22:25:10.455 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T22:25:10.478 INFO:teuthology.orchestra.run.smithi174.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-26T22:25:10.503 INFO:teuthology.orchestra.run.smithi174.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-26T22:25:10.537 INFO:teuthology.orchestra.run.smithi174.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-26T22:25:10.562 INFO:teuthology.orchestra.run.smithi174.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-26T22:25:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 606 kB/s | 51 kB 00:00 2024-04-26T22:25:10.613 INFO:teuthology.orchestra.run.smithi174.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 708 kB/s | 54 kB 00:00 2024-04-26T22:25:10.680 INFO:teuthology.orchestra.run.smithi174.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-04-26T22:25:10.705 INFO:teuthology.orchestra.run.smithi174.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 482 kB/s | 44 kB 00:00 2024-04-26T22:25:10.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T22:25:10.780 INFO:teuthology.orchestra.run.smithi174.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-04-26T22:25:10.801 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T22:25:10.805 INFO:teuthology.orchestra.run.smithi174.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-04-26T22:25:10.835 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T22:25:10.839 INFO:teuthology.orchestra.run.smithi174.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-26T22:25:10.875 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T22:25:10.923 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T22:25:10.939 INFO:teuthology.orchestra.run.smithi174.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-26T22:25:10.952 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T22:25:10.964 INFO:teuthology.orchestra.run.smithi174.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 462 kB/s | 58 kB 00:00 2024-04-26T22:25:11.028 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T22:25:11.061 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T22:25:11.104 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T22:25:11.123 INFO:teuthology.orchestra.run.smithi174.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-26T22:25:11.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T22:25:11.148 INFO:teuthology.orchestra.run.smithi174.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-04-26T22:25:11.174 INFO:teuthology.orchestra.run.smithi174.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 400 kB/s | 94 kB 00:00 2024-04-26T22:25:11.199 INFO:teuthology.orchestra.run.smithi174.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-26T22:25:11.232 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T22:25:11.276 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T22:25:11.299 INFO:teuthology.orchestra.run.smithi174.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-04-26T22:25:11.314 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T22:25:11.363 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T22:25:11.374 INFO:teuthology.orchestra.run.smithi174.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.1 MB/s | 548 kB 00:00 2024-04-26T22:25:11.425 INFO:teuthology.orchestra.run.smithi174.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-26T22:25:11.450 INFO:teuthology.orchestra.run.smithi174.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-26T22:25:11.482 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T22:25:11.492 INFO:teuthology.orchestra.run.smithi174.stdout:(107/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-26T22:25:11.517 INFO:teuthology.orchestra.run.smithi174.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-26T22:25:11.542 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T22:25:11.584 INFO:teuthology.orchestra.run.smithi174.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-26T22:25:11.609 INFO:teuthology.orchestra.run.smithi174.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 735 kB/s | 67 kB 00:00 2024-04-26T22:25:11.659 INFO:teuthology.orchestra.run.smithi174.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T22:25:11.751 INFO:teuthology.orchestra.run.smithi174.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-04-26T22:25:11.785 INFO:teuthology.orchestra.run.smithi174.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-26T22:25:11.810 INFO:teuthology.orchestra.run.smithi174.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-26T22:25:11.852 INFO:teuthology.orchestra.run.smithi174.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 359 kB/s | 15 kB 00:00 2024-04-26T22:25:11.894 INFO:teuthology.orchestra.run.smithi174.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-26T22:25:11.919 INFO:teuthology.orchestra.run.smithi174.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-26T22:25:11.970 INFO:teuthology.orchestra.run.smithi174.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T22:25:11.995 INFO:teuthology.orchestra.run.smithi174.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-26T22:25:12.037 INFO:teuthology.orchestra.run.smithi174.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-04-26T22:25:12.063 INFO:teuthology.orchestra.run.smithi174.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-26T22:25:12.121 INFO:teuthology.orchestra.run.smithi174.stdout:(122/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-04-26T22:25:12.146 INFO:teuthology.orchestra.run.smithi174.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-26T22:25:12.180 INFO:teuthology.orchestra.run.smithi174.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-26T22:25:12.205 INFO:teuthology.orchestra.run.smithi174.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-26T22:25:12.230 INFO:teuthology.orchestra.run.smithi174.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-26T22:25:12.256 INFO:teuthology.orchestra.run.smithi174.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-04-26T22:25:12.281 INFO:teuthology.orchestra.run.smithi174.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-26T22:25:12.307 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T22:25:12.323 INFO:teuthology.orchestra.run.smithi174.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-26T22:25:12.354 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T22:25:12.365 INFO:teuthology.orchestra.run.smithi174.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-04-26T22:25:12.388 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T22:25:12.407 INFO:teuthology.orchestra.run.smithi174.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-04-26T22:25:12.440 INFO:teuthology.orchestra.run.smithi174.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-04-26T22:25:12.482 INFO:teuthology.orchestra.run.smithi174.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-04-26T22:25:12.493 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T22:25:12.507 INFO:teuthology.orchestra.run.smithi174.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-26T22:25:12.533 INFO:teuthology.orchestra.run.smithi174.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-26T22:25:12.555 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T22:25:12.558 INFO:teuthology.orchestra.run.smithi174.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-26T22:25:12.583 INFO:teuthology.orchestra.run.smithi174.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-04-26T22:25:12.625 INFO:teuthology.orchestra.run.smithi174.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-26T22:25:12.646 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T22:25:12.659 INFO:teuthology.orchestra.run.smithi174.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-04-26T22:25:12.681 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T22:25:12.701 INFO:teuthology.orchestra.run.smithi174.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-04-26T22:25:12.759 INFO:teuthology.orchestra.run.smithi174.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-26T22:25:12.801 INFO:teuthology.orchestra.run.smithi174.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-04-26T22:25:12.826 INFO:teuthology.orchestra.run.smithi174.stdout:(143/152): python3-routes-2.5.1-5.el9.noarch.rp 703 kB/s | 188 kB 00:00 2024-04-26T22:25:12.866 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T22:25:12.893 INFO:teuthology.orchestra.run.smithi174.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-26T22:25:12.895 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T22:25:12.935 INFO:teuthology.orchestra.run.smithi174.stdout:(145/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-26T22:25:12.961 INFO:teuthology.orchestra.run.smithi174.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 149 kB/s | 20 kB 00:00 2024-04-26T22:25:12.975 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T22:25:12.986 INFO:teuthology.orchestra.run.smithi174.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-26T22:25:13.095 INFO:teuthology.orchestra.run.smithi174.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-26T22:25:13.570 INFO:teuthology.orchestra.run.smithi174.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 5.6 MB/s | 3.4 MB 00:00 2024-04-26T22:25:13.659 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T22:25:13.772 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T22:25:13.953 INFO:teuthology.orchestra.run.smithi174.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-04-26T22:25:14.037 INFO:teuthology.orchestra.run.smithi174.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 1.3 MB 00:00 2024-04-26T22:25:14.162 INFO:teuthology.orchestra.run.smithi174.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-04-26T22:25:14.165 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-04-26T22:25:14.165 INFO:teuthology.orchestra.run.smithi174.stdout:Total 17 MB/s | 222 MB 00:13 2024-04-26T22:25:15.075 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T22:25:15.157 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T22:25:15.185 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T22:25:15.218 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T22:25:15.274 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-04-26T22:25:15.417 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-04-26T22:25:15.417 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-04-26T22:25:15.546 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T22:25:15.589 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:25:15.680 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:25:15.710 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T22:25:15.747 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T22:25:16.252 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T22:25:16.283 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:25:16.355 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:25:16.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T22:25:17.183 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-04-26T22:25:17.184 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-04-26T22:25:18.754 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:18.789 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:19.360 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:19.401 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:25:19.495 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:25:19.495 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T22:25:19.495 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T22:25:19.495 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:19.527 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:25:23.256 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-04-26T22:25:23.324 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T22:25:23.362 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T22:25:23.424 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T22:25:23.467 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T22:25:23.492 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T22:25:23.534 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T22:25:23.855 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T22:25:23.959 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:25:24.082 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:25:24.113 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:25:24.202 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:25:24.241 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:25:24.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T22:25:24.441 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T22:25:24.465 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:25:24.552 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:25:24.574 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:25:24.636 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T22:25:24.659 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:25:24.684 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T22:25:24.858 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:25:24.891 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T22:25:24.998 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T22:25:25.094 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T22:25:25.140 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T22:25:25.174 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T22:25:25.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T22:25:25.274 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T22:25:25.317 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T22:25:25.349 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T22:25:25.377 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T22:25:25.446 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T22:25:25.499 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T22:25:25.530 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T22:25:25.565 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T22:25:25.641 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T22:25:25.676 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T22:25:25.719 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T22:25:25.748 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T22:25:25.818 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T22:25:25.871 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T22:25:25.946 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T22:25:26.090 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T22:25:26.148 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T22:25:26.679 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T22:25:26.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T22:25:26.789 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T22:25:26.832 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T22:25:26.870 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T22:25:26.920 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T22:25:27.065 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T22:25:27.115 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T22:25:27.170 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T22:25:27.242 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T22:25:27.273 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T22:25:27.310 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T22:25:27.356 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T22:25:27.393 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T22:25:27.435 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T22:25:27.477 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T22:25:27.614 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T22:25:27.658 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T22:25:27.680 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T22:25:27.721 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T22:25:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T22:25:27.846 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T22:25:27.886 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T22:25:27.912 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T22:25:27.936 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T22:25:28.014 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T22:25:28.356 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T22:25:28.385 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T22:25:28.413 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T22:25:28.447 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T22:25:28.486 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T22:25:28.518 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T22:25:28.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T22:25:28.808 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T22:25:28.841 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T22:25:28.884 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T22:25:28.917 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T22:25:29.006 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T22:25:29.056 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T22:25:29.087 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T22:25:29.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T22:25:29.232 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T22:25:29.280 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T22:25:30.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T22:25:30.052 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T22:25:30.085 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T22:25:30.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T22:25:30.251 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T22:25:30.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T22:25:30.370 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T22:25:30.539 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T22:25:30.576 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T22:25:30.654 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T22:25:31.299 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-26T22:25:31.356 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:31.416 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T22:25:31.529 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:25:31.546 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:25:31.546 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:31.546 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T22:25:31.546 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:32.644 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T22:25:32.713 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T22:25:32.741 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T22:25:32.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T22:25:33.082 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:25:33.086 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T22:25:33.109 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:25:33.110 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:33.110 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T22:25:33.110 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:25:33.110 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:25:33.110 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:33.136 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:25:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T22:25:33.258 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T22:25:33.294 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T22:25:33.341 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:25:33.365 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:33.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T22:25:33.806 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:25:33.826 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:25:33.849 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:25:33.849 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:33.850 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T22:25:33.850 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:25:33.850 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:25:33.850 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:33.877 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T22:25:33.908 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T22:25:33.964 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T22:25:34.023 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T22:25:34.086 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T22:25:34.281 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T22:25:34.326 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T22:25:34.407 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T22:25:34.497 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:25:34.639 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:25:34.676 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T22:25:34.708 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T22:25:34.735 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T22:25:35.206 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:25:35.234 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:25:35.258 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T22:25:35.321 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T22:25:35.348 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:25:36.291 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:36.329 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:36.444 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:25:36.483 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:25:36.870 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T22:25:36.915 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:25:36.995 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T22:25:36.995 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T22:25:36.995 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T22:25:36.995 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:37.042 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:25:37.498 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:25:37.534 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:25:37.643 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:25:37.811 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T22:25:37.844 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:25:37.868 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:25:37.868 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T22:25:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:25:37.869 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:25:37.869 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:37.920 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:25:37.931 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:25:38.033 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T22:25:38.094 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T22:25:39.077 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T22:25:39.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:25:39.138 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:39.181 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:25:39.204 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:25:39.204 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:39.204 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T22:25:39.205 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:39.256 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:25:39.280 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:39.599 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:25:39.622 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:25:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T22:25:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:25:39.623 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:25:39.623 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:44.486 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T22:25:44.528 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T22:25:44.568 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T22:25:44.600 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T22:25:44.627 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T22:25:44.822 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T22:25:44.865 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T22:25:44.896 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T22:25:45.547 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:25:45.592 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:25:45.613 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:25:46.524 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T22:25:46.621 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T22:25:46.663 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T22:25:46.747 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T22:25:46.775 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T22:25:46.775 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:25:46.859 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:25:46.892 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:25:46.892 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:25:46.986 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:25:47.062 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:25:48.525 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-04-26T22:25:48.526 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:48.694 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:25:48.709 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T22:25:48.709 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:48.709 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T22:25:48.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:50.164 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T22:25:50.189 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:50.414 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T22:25:50.437 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:50.881 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:25:50.904 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T22:25:50.904 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:50.904 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T22:25:50.904 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:25:50.904 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T22:25:50.905 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:51.003 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T22:25:51.051 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T22:25:51.106 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T22:25:51.298 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T22:25:51.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T22:25:51.426 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T22:25:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:25:51.654 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:25:51.706 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T22:25:51.738 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T22:25:51.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T22:25:52.219 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:25:52.248 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T22:25:52.289 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T22:25:52.349 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T22:25:52.380 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:25:52.717 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T22:25:52.718 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T22:25:52.719 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T22:25:52.731 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T22:25:52.732 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T22:25:52.733 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T22:25:52.734 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T22:25:52.735 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T22:25:52.736 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T22:25:52.737 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T22:25:52.738 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:25:52.739 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T22:25:53.410 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T22:25:53.448 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:25:54.407 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T22:25:54.440 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:25:54.541 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T22:25:54.702 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T22:25:54.733 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T22:25:54.757 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:54.801 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:25:54.812 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T22:25:54.904 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:25:54.904 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:54.904 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-26T22:25:54.904 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.904 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.905 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.906 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T22:25:54.907 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T22:25:54.908 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T22:25:54.909 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.909 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.909 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T22:25:54.909 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T22:25:54.910 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.911 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T22:25:54.912 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T22:25:54.913 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T22:25:54.914 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T22:25:54.915 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T22:25:54.916 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T22:25:54.917 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T22:25:54.917 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T22:25:54.917 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T22:25:54.917 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:25:54.917 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-26T22:25:54.938 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T22:25:54.991 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T22:25:55.323 DEBUG:teuthology.parallel:result is None 2024-04-26T22:25:55.918 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T22:25:55.954 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T22:25:55.976 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:56.021 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:25:56.042 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T22:25:56.042 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:56.042 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T22:25:56.042 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:56.088 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:25:56.110 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T22:25:56.111 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:56.111 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T22:25:56.111 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:25:56.111 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T22:25:56.111 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:25:56.413 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T22:25:56.436 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:01.017 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T22:26:01.058 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T22:26:01.099 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T22:26:01.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T22:26:01.158 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T22:26:01.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T22:26:01.379 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T22:26:01.417 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T22:26:02.030 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:26:02.073 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:26:02.093 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T22:26:02.907 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T22:26:02.994 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T22:26:03.060 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T22:26:03.141 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T22:26:03.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T22:26:03.173 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:26:03.257 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T22:26:03.290 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:26:03.290 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:26:03.434 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T22:26:03.505 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:26:09.111 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:26:09.111 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T22:26:09.112 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T22:26:09.113 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T22:26:09.114 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T22:26:09.126 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T22:26:09.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T22:26:09.128 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T22:26:09.129 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T22:26:09.130 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T22:26:09.131 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T22:26:09.132 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T22:26:09.133 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T22:26:09.134 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T22:26:09.135 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T22:26:11.232 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.233 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.234 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T22:26:11.235 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T22:26:11.236 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T22:26:11.237 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T22:26:11.238 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T22:26:11.239 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T22:26:11.240 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T22:26:11.241 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.242 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T22:26:11.243 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T22:26:11.244 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T22:26:11.245 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T22:26:11.245 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:11.245 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-04-26T22:26:11.702 DEBUG:teuthology.parallel:result is None 2024-04-26T22:26:11.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:26:11.921 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T22:26:11.947 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T22:26:11.948 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T22:26:11.948 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T22:26:11.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T22:26:12.170 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T22:26:12.191 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T22:26:12.191 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T22:26:12.192 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T22:26:12.193 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T22:26:12.194 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:12.194 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T22:26:12.226 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:12.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T22:26:12.257 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T22:26:12.258 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:12.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T22:26:12.295 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T22:26:12.362 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:12.363 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T22:26:12.386 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T22:26:12.450 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T22:26:12.450 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:12.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T22:26:12.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T22:26:12.545 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:12.545 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T22:26:12.568 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T22:26:12.631 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T22:26:12.632 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:12.632 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T22:26:12.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T22:26:12.720 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:12.720 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T22:26:12.742 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T22:26:12.806 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T22:26:12.893 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T22:26:12.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /var/log/ceph 2024-04-26T22:26:12.895 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /var/log/ceph 2024-04-26T22:26:12.918 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T22:26:12.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T22:26:12.960 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T22:26:12.983 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T22:26:12.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T22:26:13.025 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T22:26:13.055 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T22:26:13.055 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T22:26:13.056 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T22:26:13.056 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:13.056 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T22:26:13.103 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:13.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:13.238 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:13.238 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T22:26:13.253 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:13.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:13.404 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T22:26:13.404 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T22:26:13.404 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675262', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7675262', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi062.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObeXS9mii0wDMCFiv+X9R2zlzi8Rs8AhhIuJ1ApD/9aLbUfFWoc/YYibC2pW1/Rk8RhkHtYtO0U2dOUVWrsKRw=', 'smithi174.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOjdXFT3YZ+ISz+3bGJFTCuOwtbluhNFlDd4BHFxuurJOc0kGD0IqPDSyoqN411sdH/p1cA71q6wJ0db5xAWskI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T22:26:13.404 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T22:26:13.407 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T22:26:13.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T22:26:13.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T22:26:13.516 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:13.516 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T22:26:13.539 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T22:26:13.539 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-26 22:25:47.454659904 +0000 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-26 22:24:02.417327779 +0000 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-26 22:24:02.417327779 +0000 2024-04-26T22:26:13.594 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-26 22:24:02.417327779 +0000 2024-04-26T22:26:13.594 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T22:26:13.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-26T22:26:13.656 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-26T22:26:13.656 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000243575 s, 2.1 MB/s 2024-04-26T22:26:13.658 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T22:26:13.713 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-26T22:26:13.769 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T22:26:13.769 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-26 22:25:47.454659904 +0000 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-26 22:24:02.779322502 +0000 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-26 22:24:02.779322502 +0000 2024-04-26T22:26:13.770 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-26 22:24:02.779322502 +0000 2024-04-26T22:26:13.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T22:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-26T22:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-26T22:26:13.833 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000178325 s, 2.9 MB/s 2024-04-26T22:26:13.835 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T22:26:13.891 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-26 22:25:47.455659890 +0000 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-26 22:24:03.132317357 +0000 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-26 22:24:03.132317357 +0000 2024-04-26T22:26:13.947 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-26 22:24:03.132317357 +0000 2024-04-26T22:26:13.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T22:26:14.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-26T22:26:14.009 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-26T22:26:14.009 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000181939 s, 2.8 MB/s 2024-04-26T22:26:14.010 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T22:26:14.067 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-26T22:26:14.121 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-26 22:25:47.455659890 +0000 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-26 22:24:03.472312401 +0000 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-26 22:24:03.472312401 +0000 2024-04-26T22:26:14.122 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-26 22:24:03.472312401 +0000 2024-04-26T22:26:14.122 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T22:26:14.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-26T22:26:14.184 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-26T22:26:14.184 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000203803 s, 2.5 MB/s 2024-04-26T22:26:14.185 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T22:26:14.243 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:14.243 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:14.243 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T22:26:14.258 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T22:26:14.258 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 22:26:03.867563551 +0000 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 22:23:54.315852192 +0000 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 22:23:54.315852192 +0000 2024-04-26T22:26:14.313 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 22:23:54.315852192 +0000 2024-04-26T22:26:14.314 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T22:26:14.375 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T22:26:14.375 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T22:26:14.375 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000249402 s, 2.1 MB/s 2024-04-26T22:26:14.376 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T22:26:14.431 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 22:26:03.868563537 +0000 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 22:23:54.664847173 +0000 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 22:23:54.664847173 +0000 2024-04-26T22:26:14.486 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 22:23:54.664847173 +0000 2024-04-26T22:26:14.487 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T22:26:14.548 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T22:26:14.548 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T22:26:14.548 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000217762 s, 2.4 MB/s 2024-04-26T22:26:14.549 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T22:26:14.604 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 22:26:03.868563537 +0000 2024-04-26T22:26:14.659 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 22:23:55.001842326 +0000 2024-04-26T22:26:14.660 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 22:23:55.001842326 +0000 2024-04-26T22:26:14.660 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 22:23:55.001842326 +0000 2024-04-26T22:26:14.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T22:26:14.721 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T22:26:14.721 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T22:26:14.721 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000203509 s, 2.5 MB/s 2024-04-26T22:26:14.722 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T22:26:14.777 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 22:26:03.868563537 +0000 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 22:23:55.344837393 +0000 2024-04-26T22:26:14.832 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 22:23:55.344837393 +0000 2024-04-26T22:26:14.833 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 22:23:55.344837393 +0000 2024-04-26T22:26:14.833 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T22:26:14.894 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T22:26:14.894 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T22:26:14.895 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000223277 s, 2.3 MB/s 2024-04-26T22:26:14.896 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T22:26:14.952 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:14.952 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T22:26:14.952 INFO:tasks.ceph:Generating config... 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] pid file = 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T22:26:14.953 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T22:26:14.953 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T22:26:14.954 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T22:26:14.955 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T22:26:14.955 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T22:26:14.955 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T22:26:14.955 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T22:26:14.955 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T22:26:14.956 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T22:26:14.956 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T22:26:14.956 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T22:26:14.956 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T22:26:14.956 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T22:26:14.956 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T22:26:14.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T22:26:15.022 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T22:26:15.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T22:26:15.069 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T22:26:15.094 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.62'), ('mon.b', '172.21.15.174'), ('mon.c', '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]')] 2024-04-26T22:26:15.095 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.62,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T22:26:15.095 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:15.095 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T22:26:15.151 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.62 --add b 172.21.15.174 --addv c '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:15.237 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: generated fsid f70b0a7c-519a-45f1-aa13-a1f2de1080a4 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:setting min_mon_release = quincy 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:epoch 0 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:fsid f70b0a7c-519a-45f1-aa13-a1f2de1080a4 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:last_changed 2024-04-26T22:26:15.238452+0000 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:created 2024-04-26T22:26:15.238452+0000 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:min_mon_release 17 (quincy) 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:election_strategy: 1 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:0: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.a 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2024-04-26T22:26:15.238 INFO:teuthology.orchestra.run.smithi062.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T22:26:15.239 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T22:26:15.254 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f70b0a7c-519a-45f1-aa13-a1f2de1080a4... 2024-04-26T22:26:15.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T22:26:15.297 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T22:26:15.336 INFO:teuthology.orchestra.run.smithi062.stdout:[global] 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: chdir = "" 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: auth supported = cephx 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: filestore xattr use omap = true 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: mon clock drift allowed = 1.000 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: osd crush chooseleaf type = 0 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: auth debug = true 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on old message = true 2024-04-26T22:26:15.337 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on bug = true 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon pg warn max object skew = 0 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default size = 2 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow primary affinity = true 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd allow pg remap = true 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on legacy crush tunables = false 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on crush straw calc version zero = false 2024-04-26T22:26:15.338 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on no sortbitwise = false 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on osd down out interval zero = false 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon warn on too few osds = false 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon_allow_pool_size_one = true 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: osd default data pool replay window = 5 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon allow pool delete = true 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon cluster log file level = debug 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: debug asserts on shutdown = true 2024-04-26T22:26:15.339 INFO:teuthology.orchestra.run.smithi062.stdout: mon health detail to clog = false 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: mon host = "172.21.15.62,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: ms die on skipped message = False 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: fsid = f70b0a7c-519a-45f1-aa13-a1f2de1080a4 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout:[osd] 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd journal size = 100 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub load threshold = 5.0 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd scrub max interval = 600 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd mclock profile = high_recovery_ops 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd recover clone overlap = true 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: osd recovery max chunk = 1048576 2024-04-26T22:26:15.340 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug shutdown = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug op order = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug verify stray on activate = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug trim objects = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd open classes on start = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug pg log writeout = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd deep scrub update digest min age = 30 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: osd map max advance = 10 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: journal zero on create = true 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.341 INFO:teuthology.orchestra.run.smithi062.stdout: filestore ondisk finisher threads = 3 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: filestore apply finisher threads = 3 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bdev debug aio = true 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: osd debug misdirected ops = true 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bdev async discard = True 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bdev enable discard = True 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore allocator = bitmap 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore block size = 96636764160 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: bluestore fsck on mount = True 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluefs = 1/20 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: debug bluestore = 1/20 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: debug osd = 20 2024-04-26T22:26:15.342 INFO:teuthology.orchestra.run.smithi062.stdout: debug rocksdb = 4/10 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd full ratio = 0.9 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd nearfull ratio = 0.8 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: osd failsafe full ratio = 0.95 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: osd objectstore = bluestore 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr] 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: debug mgr = 20 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-26T22:26:15.343 INFO:teuthology.orchestra.run.smithi062.stdout: mgr/telemetry/nag = false 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug finisher = 20 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout:[mon] 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug mon = 20 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug paxos = 20 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: debug auth = 20 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon data avail warn = 5 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon mgr mkfs grace = 240 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min pgs per osd = 4 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd reporter subtree level = osd 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon osd prime pg temp = true 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: mon reweight min bytes per osd = 10 2024-04-26T22:26:15.344 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: # 1m isn't quite enough 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon_down_mkfs_grace = 2m 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon_warn_on_filestore_osds = false 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon down mkfs grace = 300 2024-04-26T22:26:15.345 INFO:teuthology.orchestra.run.smithi062.stdout: mon op complaint time = 120 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout:[client] 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: rgw cache enabled = true 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable ops log = true 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: rgw enable usage log = true 2024-04-26T22:26:15.346 INFO:teuthology.orchestra.run.smithi062.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: client mount timeout = 600 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: debug client = 20 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: pid file = "" 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.a] 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.b] 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mon.c] 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-26T22:26:15.350 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout:[mds] 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds = 20 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: debug mds balancer = 20 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: debug ms = 1 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug frag = True 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: mds debug scatterstat = True 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: mds op complaint time = 180 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: mds verify scatter = True 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: osd op complaint time = 180 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: rados mon op timeout = 900 2024-04-26T22:26:15.351 INFO:teuthology.orchestra.run.smithi062.stdout: rados osd op timeout = 900 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.353 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2024-04-26T22:26:15.354 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2024-04-26T22:26:15.355 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.62,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on skipped message = False 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = f70b0a7c-519a-45f1-aa13-a1f2de1080a4 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2024-04-26T22:26:15.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug trim objects = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.357 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore allocator = bitmap 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1/20 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1/20 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2024-04-26T22:26:15.358 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 4/10 2024-04-26T22:26:15.359 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T22:26:15.359 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2024-04-26T22:26:15.359 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2024-04-26T22:26:15.359 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2024-04-26T22:26:15.359 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug finisher = 20 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2024-04-26T22:26:15.360 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.361 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon down mkfs grace = 300 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: mon op complaint time = 120 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2024-04-26T22:26:15.362 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: client mount timeout = 600 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = "" 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout:[mds] 2024-04-26T22:26:15.363 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds = 20 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds balancer = 20 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug frag = True 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug scatterstat = True 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: mds op complaint time = 180 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: mds verify scatter = True 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2024-04-26T22:26:15.364 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2024-04-26T22:26:15.364 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T22:26:15.364 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T22:26:15.446 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T22:26:15.447 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:15.447 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T22:26:15.462 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:15.462 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T22:26:15.517 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:26:15.517 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:15.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T22:26:15.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T22:26:15.590 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:15.590 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:15.646 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:26:15.646 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:15.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T22:26:15.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T22:26:15.678 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:15.679 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:15.733 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T22:26:15.733 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T22:26:15.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T22:26:15.783 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T22:26:15.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T22:26:15.848 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T22:26:15.850 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T22:26:15.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T22:26:15.900 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T22:26:15.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T22:26:15.927 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T22:26:16.018 INFO:teuthology.orchestra.run.smithi062.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T22:26:16.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T22:26:16.047 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T22:26:16.107 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T22:26:16.107 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T22:26:16.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T22:26:16.277 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T22:26:16.278 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T22:26:16.307 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T22:26:16.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T22:26:16.350 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T22:26:16.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T22:26:16.446 INFO:teuthology.orchestra.run.smithi062.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T22:26:16.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T22:26:16.499 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T22:26:16.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T22:26:16.590 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T22:26:16.602 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T22:26:16.602 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi062.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T22:26:16.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T22:26:16.628 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:16.628 INFO:tasks.ceph:role: osd.0 2024-04-26T22:26:16.628 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:26:16.628 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:16.701 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:16.702 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:17.216 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-26T22:26:17.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-26T22:26:17.218 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T22:26:17.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T22:26:17.356 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T22:26:17.423 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:17.423 INFO:tasks.ceph:role: osd.1 2024-04-26T22:26:17.423 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:26:17.423 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:17.495 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:17.496 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:17.963 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-26T22:26:17.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-26T22:26:17.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T22:26:18.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T22:26:18.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T22:26:18.152 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:18.152 INFO:tasks.ceph:role: osd.2 2024-04-26T22:26:18.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:26:18.152 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:18.224 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:18.725 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-26T22:26:18.727 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-26T22:26:18.727 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T22:26:18.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T22:26:18.894 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T22:26:18.961 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:18.961 INFO:tasks.ceph:role: osd.3 2024-04-26T22:26:18.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:26:18.961 DEBUG:teuthology.orchestra.run.smithi062:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:19.036 INFO:teuthology.orchestra.run.smithi062.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:19.547 INFO:teuthology.orchestra.run.smithi062.stdout:Discarding blocks...Done. 2024-04-26T22:26:19.549 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi062.front.sepia.ceph.com -o noatime 2024-04-26T22:26:19.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T22:26:19.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T22:26:19.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.801+0000 7fc3cf217640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T22:26:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.802+0000 7fc3cf217640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T22:26:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.802+0000 7fc3cf217640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T22:26:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.802+0000 7fc3cf217640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T22:26:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.802+0000 7fc3cf217640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T22:26:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:19.802+0000 7fc3cf217640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T22:26:22.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T22:26:22.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.782+0000 7fce73232640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.783+0000 7fce73232640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.783+0000 7fce73232640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.783+0000 7fce73232640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.783+0000 7fce73232640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T22:26:22.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:22.783+0000 7fce73232640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T22:26:25.626 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T22:26:25.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.747+0000 7fd260063640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.748+0000 7fd260063640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.748+0000 7fd260063640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.748+0000 7fd260063640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.748+0000 7fd260063640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T22:26:25.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:25.748+0000 7fd260063640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T22:26:28.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T22:26:28.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:28.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.726+0000 7fee64e14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T22:26:28.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.726+0000 7fee64e14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T22:26:28.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.726+0000 7fee64e14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T22:26:28.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.726+0000 7fee64e14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T22:26:28.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.726+0000 7fee64e14640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T22:26:28.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:28.727+0000 7fee64e14640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T22:26:31.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T22:26:31.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T22:26:31.646 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:31.646 INFO:tasks.ceph:role: osd.4 2024-04-26T22:26:31.646 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:26:31.646 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T22:26:31.724 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:31.725 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:32.228 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-04-26T22:26:32.230 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-04-26T22:26:32.230 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T22:26:32.352 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T22:26:32.384 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T22:26:32.448 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:32.448 INFO:tasks.ceph:role: osd.5 2024-04-26T22:26:32.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:26:32.448 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T22:26:32.519 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:32.520 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:32.989 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-04-26T22:26:32.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-04-26T22:26:32.991 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T22:26:33.076 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T22:26:33.102 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T22:26:33.165 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:33.165 INFO:tasks.ceph:role: osd.6 2024-04-26T22:26:33.165 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:26:33.166 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:33.237 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:33.237 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:33.237 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:33.728 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-04-26T22:26:33.730 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-04-26T22:26:33.730 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T22:26:33.854 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T22:26:33.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T22:26:33.946 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T22:26:33.946 INFO:tasks.ceph:role: osd.7 2024-04-26T22:26:33.946 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:26:33.946 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T22:26:34.017 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T22:26:34.018 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T22:26:34.521 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-04-26T22:26:34.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-04-26T22:26:34.524 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T22:26:34.677 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T22:26:34.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.819+0000 7f22ce418640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T22:26:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.819+0000 7f22ce418640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T22:26:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.819+0000 7f22ce418640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T22:26:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.819+0000 7f22ce418640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T22:26:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.819+0000 7f22ce418640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T22:26:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:34.820+0000 7f22ce418640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T22:26:37.674 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T22:26:37.702 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:37.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.793+0000 7f534d60d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T22:26:37.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.793+0000 7f534d60d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T22:26:37.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.793+0000 7f534d60d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T22:26:37.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.793+0000 7f534d60d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T22:26:37.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.793+0000 7f534d60d640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T22:26:37.794 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:37.794+0000 7f534d60d640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T22:26:40.645 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T22:26:40.675 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:40.765 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.765+0000 7ff692a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T22:26:40.765 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.765+0000 7ff692a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T22:26:40.765 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.765+0000 7ff692a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T22:26:40.765 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.765+0000 7ff692a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T22:26:40.765 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.765+0000 7ff692a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T22:26:40.766 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:40.766+0000 7ff692a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T22:26:43.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:43.409 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:26:43.627 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T22:26:43.652 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:43.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.736+0000 7ff8c9a19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T22:26:43.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.736+0000 7ff8c9a19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T22:26:43.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.736+0000 7ff8c9a19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T22:26:43.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.736+0000 7ff8c9a19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T22:26:43.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.736+0000 7ff8c9a19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T22:26:43.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:43.737+0000 7ff8c9a19640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T22:26:46.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T22:26:46.659 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T22:26:46.659 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.659 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T22:26:46.689 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T22:26:46.757 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T22:26:46.823 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T22:26:46.890 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T22:26:46.956 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:46.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T22:26:47.021 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:47.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T22:26:47.086 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.086 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T22:26:47.114 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.114 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T22:26:47.180 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.180 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T22:26:47.247 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.248 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T22:26:47.312 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.312 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T22:26:47.377 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.377 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T22:26:47.442 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.442 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T22:26:47.506 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:47.506 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T22:26:47.523 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:47.524 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T22:26:47.581 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.581 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T22:26:47.597 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:47.597 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T22:26:47.652 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T22:26:47.652 DEBUG:teuthology.orchestra.run.smithi062:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T22:26:47.655 DEBUG:teuthology.orchestra.run.smithi174:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.y] 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAHKixm0y23LhAA5KnsIydYZchjnmyUwLWl1w== 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.a] 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAHKixmWg+9NRAAdnpMVpgAaGZ21j6W0Tbs+w== 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.c] 2024-04-26T22:26:47.695 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixm/wIqARAA55sdB593n/Skd4uA1syuSg== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.0] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQALKixmMfTWLxAAg/vI2T8zb3xX7r3ZgdrTFg== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.1] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAOKixmfyK2LhAAHbPw2zCa8HuUWqsXiFPkPQ== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.2] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQARKixmSo+dLBAAiPoVs57GokukpcqH47rfAw== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.3] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAUKixmXiJYKxAAo5iZwxDjIP6rn0cRysJxfg== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[mgr.x] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAHKixmgseXMhAAtwa5QiDplGCQk5DQlFHHvg== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.b] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmwQbLBRAAuP30XZDLsJaBm6BNHTqFCA== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[mds.d] 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixm0jNbEBAAudci4vHH/MXyeG7tCWL1Nw== 2024-04-26T22:26:47.696 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.4] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAaKixmVSbbMBAA4oChxHAndPdjzYm3K59Acg== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.5] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAdKixmTA5LLxAAtirJ58cd12oGcGYZ4W4eQg== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.6] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAgKixm3+KjLRAAjf0ELmtEHU5hdTojxCPJDw== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[osd.7] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAjKixmm0jnKxAA7IYEt06xhis1drk5h0zH2w== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[client.2] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmZOyjGhAAxeZlr2qggbHBj9ZjmCsOqA== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[client.1] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixm20S/HRAAlRqwTgIuwG+sjvnjS87R1g== 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout:[client.3] 2024-04-26T22:26:47.697 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmfI08IxAAbwPyTVFRY5J5Vw08V31Ocw== 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.y] 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAHKixm0y23LhAA5KnsIydYZchjnmyUwLWl1w== 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAHKixmWg+9NRAAdnpMVpgAaGZ21j6W0Tbs+w== 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.c] 2024-04-26T22:26:47.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixm/wIqARAA55sdB593n/Skd4uA1syuSg== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.0] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQALKixmMfTWLxAAg/vI2T8zb3xX7r3ZgdrTFg== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.1] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAOKixmfyK2LhAAHbPw2zCa8HuUWqsXiFPkPQ== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.2] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQARKixmSo+dLBAAiPoVs57GokukpcqH47rfAw== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.3] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAUKixmXiJYKxAAo5iZwxDjIP6rn0cRysJxfg== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.x] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAHKixmgseXMhAAtwa5QiDplGCQk5DQlFHHvg== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixmwQbLBRAAuP30XZDLsJaBm6BNHTqFCA== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.d] 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixm0jNbEBAAudci4vHH/MXyeG7tCWL1Nw== 2024-04-26T22:26:47.715 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.4] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAaKixmVSbbMBAA4oChxHAndPdjzYm3K59Acg== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.5] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAdKixmTA5LLxAAtirJ58cd12oGcGYZ4W4eQg== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.6] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAgKixm3+KjLRAAjf0ELmtEHU5hdTojxCPJDw== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.7] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAjKixmm0jnKxAA7IYEt06xhis1drk5h0zH2w== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[client.2] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixmZOyjGhAAxeZlr2qggbHBj9ZjmCsOqA== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[client.1] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixm20S/HRAAlRqwTgIuwG+sjvnjS87R1g== 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout:[client.3] 2024-04-26T22:26:47.716 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAIKixmfI08IxAAbwPyTVFRY5J5Vw08V31Ocw== 2024-04-26T22:26:47.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T22:26:47.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T22:26:47.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:47.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:47.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:47.864 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:47.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:47.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:47.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:47.970 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.068 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.071 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T22:26:48.122 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T22:26:48.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:48.172 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:48.224 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:48.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T22:26:48.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.281 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.331 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.382 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.434 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T22:26:48.483 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.540 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T22:26:48.699 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T22:26:48.699 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T22:26:48.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T22:26:48.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T22:26:48.843 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T22:26:48.867 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T22:26:48.960 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T22:26:48.984 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T22:26:49.051 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T22:26:49.145 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T22:26:49.168 DEBUG:teuthology.orchestra.run.smithi062:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:49.170 DEBUG:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T22:26:49.223 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T22:26:49.223 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T22:26:49.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T22:26:49.226 INFO:tasks.ceph.mon.a:Started 2024-04-26T22:26:49.226 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T22:26:49.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T22:26:49.266 INFO:tasks.ceph.mon.b:Started 2024-04-26T22:26:49.266 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T22:26:49.266 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T22:26:49.268 INFO:tasks.ceph.mon.c:Started 2024-04-26T22:26:49.268 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T22:26:49.268 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T22:26:49.269 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T22:26:49.271 INFO:tasks.ceph.mgr.y:Started 2024-04-26T22:26:49.271 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T22:26:49.271 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T22:26:49.273 INFO:tasks.ceph.mgr.x:Started 2024-04-26T22:26:49.273 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T22:26:49.273 DEBUG:teuthology.orchestra.run.smithi062:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T22:26:49.373 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.372+0000 7febb190a640 1 Processor -- start 2024-04-26T22:26:49.374 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.372+0000 7febb190a640 1 -- start start 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febb190a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7febac0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febb190a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac0b5fd0 con 0x7febac0b5510 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febb190a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febac0d5c80 con 0x7febac0d1fa0 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febb190a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7febac0b5d80 con 0x7febac0b6820 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaaffd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7febac0b5510 0x7febac0d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35082/0 (socket says 172.21.15.62:35082) 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaaffd640 1 -- 172.21.15.62:0/1253112391 learned_addr learned my addr 172.21.15.62:0/1253112391 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7febac0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaa7fc640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 msgr2=0x7febac0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T22:26:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaa7fc640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 msgr2=0x7febac0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T22:26:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7febac0c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T22:26:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.373+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7febac0c8cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T22:26:49.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.528+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897222961 0 0) 0x7febac0b5fd0 con 0x7febac0b5510 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.528+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb94003520 con 0x7febac0b5510 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878069207 0 0) 0x7febac0d5c80 con 0x7febac0d1fa0 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febac0b5fd0 con 0x7febac0d1fa0 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3769059457 0 0) 0x7feb94003520 con 0x7febac0b5510 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 msgr2=0x7febac0c8cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7febac0c8cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 >> v1:172.21.15.174:6789/0 conn(0x7febac0d1fa0 legacy=0x7febac0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0d5f60 con 0x7febac0b5510 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7feba0001670 con 0x7febac0b5510 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 >> v1:172.21.15.62:6789/0 conn(0x7febac0b5510 legacy=0x7febac0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7feb94003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb94004120 0x7feb94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004d40 0x7feb940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febac0b5d80 con 0x7feb94004120 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb9400bc20 con 0x7feb94004d40 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.529+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feb9400be20 con 0x7febac0b6820 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febab7fe640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004d40 0x7feb940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb94004120 0x7feb94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaaffd640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7feb94003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 msgr2=0x7feb94003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7feb94003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004d40 msgr2=0x7feb940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004d40 0x7feb940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaa7fc640 1 -- 172.21.15.62:0/1253112391 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb94003520 con 0x7feb94004120 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febaaffd640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0b6820 0x7feb94003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:49.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.530+0000 7febab7fe640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004d40 0x7feb940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.531+0000 7febaa7fc640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb94004120 0x7feb94004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b9e0 tx=0x7feb94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d53f011ae37859 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.531+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c030070 con 0x7feb94004120 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.531+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c02ada0 con 0x7feb94004120 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.531+0000 7feba9ffb640 1 -- 172.21.15.62:0/1253112391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00b5e0 con 0x7feb94004120 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 -- 172.21.15.62:0/1253112391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb94004120 msgr2=0x7feb94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 --2- 172.21.15.62:0/1253112391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb94004120 0x7feb94004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00b9e0 tx=0x7feb94005830 comp rx=0 tx=0).stop 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febab7fe640 1 -- 172.21.15.62:0/1253112391 reap_dead start 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 -- 172.21.15.62:0/1253112391 shutdown_connections 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 -- 172.21.15.62:0/1253112391 >> 172.21.15.62:0/1253112391 conn(0x7febac0d5160 msgr2=0x7febac0d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 -- 172.21.15.62:0/1253112391 shutdown_connections 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.532+0000 7febb190a640 1 -- 172.21.15.62:0/1253112391 wait complete. 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.533+0000 7febb190a640 1 Processor -- start 2024-04-26T22:26:49.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.534+0000 7febb190a640 1 -- start start 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 0x7febac15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 0x7febac1e9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0ce130 0x7febac0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d4100 con 0x7febac1e9310 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febac0d0b60 con 0x7feb94004120 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febb190a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7febac0d1080 con 0x7febac0ce130 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 0x7febac15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 0x7febac1e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 0x7febac15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:58386/0 (socket says 172.21.15.62:58386) 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaaffd640 1 -- 172.21.15.62:0/3085789721 learned_addr learned my addr 172.21.15.62:0/3085789721 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febab7fe640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0ce130 0x7febac0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:49.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 -- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0ce130 msgr2=0x7febac0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0ce130 0x7febac0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 -- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 msgr2=0x7febac15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 0x7febac15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 -- 172.21.15.62:0/3085789721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac0cc2d0 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.535+0000 7febaa7fc640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 0x7febac1e9790 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb9c032000 tx=0x7feb9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c043020 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c03f680 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac0cc5b0 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00b5e0 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac0ccb70 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb9c046020 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.536+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb9c03db50 con 0x7febac1e9310 2024-04-26T22:26:49.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.537+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febac0693e0 con 0x7febac1e9310 2024-04-26T22:26:49.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.539+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7feb9c04d030 con 0x7febac1e9310 2024-04-26T22:26:49.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.606+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7febac0cdf10 con 0x7febac1e9310 2024-04-26T22:26:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.607+0000 7feb93fff640 1 -- 172.21.15.62:0/3085789721 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7feb9c03dd50 con 0x7febac1e9310 2024-04-26T22:26:49.608 INFO:teuthology.orchestra.run.smithi062.stderr:adjusted tunables profile to default 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 msgr2=0x7febac1e9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 0x7febac1e9790 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb9c032000 tx=0x7feb9c004730 comp rx=0 tx=0).stop 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 shutdown_connections 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7febac0ce130 0x7febac0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7febac1e9310 0x7febac1e9790 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 --2- 172.21.15.62:0/3085789721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb94004120 0x7febac15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 >> 172.21.15.62:0/3085789721 conn(0x7febac0d5160 msgr2=0x7febac0d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 shutdown_connections 2024-04-26T22:26:49.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:49.608+0000 7febb190a640 1 -- 172.21.15.62:0/3085789721 wait complete. 2024-04-26T22:26:49.620 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T22:26:49.620 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T22:26:49.620 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:49.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T22:26:49.635 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:49.634+0000 7f9d3d61e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:26:49.644 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:49.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T22:26:49.652 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:49.651+0000 7f5eb4012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T22:26:49.676 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:49.676 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T22:26:49.740 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:26:49.740 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T22:26:49.781 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:49.781+0000 7f9d3d61e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:26:49.804 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:49.804 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T22:26:49.810 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:49.809+0000 7f5eb4012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:26:49.828 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:49.828 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T22:26:49.836 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:49.836+0000 7f9d3d61e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:26:49.859 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:49.859 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T22:26:49.866 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:49.865+0000 7f5eb4012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T22:26:49.923 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:26:49.923 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T22:26:49.987 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 49e0d6ca-021d-48d2-8b0c-c00f30017d21 0 2024-04-26T22:26:50.093 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:50.092+0000 7f9d3d61e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:26:50.121 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:50.120+0000 7f5eb4012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.124+0000 7f2ac6ad1640 1 Processor -- start 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.124+0000 7f2ac6ad1640 1 -- start start 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 0x7f2ac00d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 0x7f2ac00d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2ac00cf990 0x7f2ac00cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac00ceeb0 con 0x7f2ac00cf990 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac015d700 con 0x7f2ac00d1090 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac6ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac00ce8f0 con 0x7f2ac015d320 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2abffff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 0x7f2ac00d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.127 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 0x7f2ac00d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2abffff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 0x7f2ac00d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43660/0 (socket says 172.21.15.174:43660) 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2abffff640 1 -- 172.21.15.174:0/142483500 learned_addr learned my addr 172.21.15.174:0/142483500 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac506e640 1 --2- 172.21.15.174:0/142483500 >> v2:172.21.15.62:3300/0 conn(0x7f2ac00cf990 0x7f2ac00cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 -- 172.21.15.174:0/142483500 >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 msgr2=0x7f2ac00d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 --2- 172.21.15.174:0/142483500 >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 0x7f2ac00d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 -- 172.21.15.174:0/142483500 >> v2:172.21.15.62:3300/0 conn(0x7f2ac00cf990 msgr2=0x7f2ac00cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 --2- 172.21.15.174:0/142483500 >> v2:172.21.15.62:3300/0 conn(0x7f2ac00cf990 0x7f2ac00cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.125+0000 7f2ac486d640 1 -- 172.21.15.174:0/142483500 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac00cebd0 con 0x7f2ac015d320 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.126+0000 7f2ac486d640 1 --2- 172.21.15.174:0/142483500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 0x7f2ac00d0b50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00738f0 tx=0x7f2ab402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86c96132032caa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.126+0000 7f2abf7fe640 1 -- 172.21.15.174:0/142483500 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4037070 con 0x7f2ac015d320 2024-04-26T22:26:50.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.126+0000 7f2abf7fe640 1 -- 172.21.15.174:0/142483500 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab402add0 con 0x7f2ac015d320 2024-04-26T22:26:50.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.126+0000 7f2abf7fe640 1 -- 172.21.15.174:0/142483500 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4032660 con 0x7f2ac015d320 2024-04-26T22:26:50.132 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/142483500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 msgr2=0x7f2ac00d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.132 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/142483500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 0x7f2ac00d0b50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00738f0 tx=0x7f2ab402a080 comp rx=0 tx=0).stop 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/142483500 shutdown_connections 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/142483500 >> v2:172.21.15.62:3300/0 conn(0x7f2ac00cf990 0x7f2ac00cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/142483500 >> v2:172.21.15.174:3300/0 conn(0x7f2ac00d1090 0x7f2ac00d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/142483500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac015d320 0x7f2ac00d0b50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/142483500 >> 172.21.15.174:0/142483500 conn(0x7f2ac00d2f80 msgr2=0x7f2ac00d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/142483500 shutdown_connections 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.127+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/142483500 wait complete. 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.128+0000 7f2ac6ad1640 1 Processor -- start 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.128+0000 7f2ac6ad1640 1 -- start start 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 0x7f2ac01f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ac015d320 0x7f2ac0154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00ce550 con 0x7f2ac015d320 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00b2cd0 con 0x7f2ac00cf990 2024-04-26T22:26:50.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac6ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac00ce6d0 con 0x7f2ac00d1090 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac486d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 0x7f2ac01f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57866/0 (socket says 172.21.15.174:57866) 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 -- 172.21.15.174:0/2463055107 learned_addr learned my addr 172.21.15.174:0/2463055107 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac506e640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ac015d320 0x7f2ac0154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 -- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 msgr2=0x7f2ac01f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 0x7f2ac01f34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 -- 172.21.15.174:0/2463055107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ac015d320 msgr2=0x7f2ac0154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ac015d320 0x7f2ac0154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 -- 172.21.15.174:0/2463055107 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac0159480 con 0x7f2ac00d1090 2024-04-26T22:26:50.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2ac486d640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 0x7f2ac01f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.129+0000 7f2abffff640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e840 tx=0x7f2aac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac017070 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aac00c8e0 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac0159760 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00ccd0 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac0159d20 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2aac01a070 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.130+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f2aac01c7d0 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.131+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac00693e0 con 0x7f2ac00d1090 2024-04-26T22:26:50.136 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.133+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2aac013470 con 0x7f2ac00d1090 2024-04-26T22:26:50.141 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:50.141+0000 7f9d3d61e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:26:50.169 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:50.168+0000 7f5eb4012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T22:26:50.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.205+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "49e0d6ca-021d-48d2-8b0c-c00f30017d21", "id": 0} v 0) -- 0x7f2ac00cd670 con 0x7f2ac00d1090 2024-04-26T22:26:50.209 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2abd7fa640 1 -- 172.21.15.174:0/2463055107 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49e0d6ca-021d-48d2-8b0c-c00f30017d21", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f2aac01ca70 con 0x7f2ac00d1090 2024-04-26T22:26:50.209 INFO:teuthology.orchestra.run.smithi174.stdout:0 2024-04-26T22:26:50.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 msgr2=0x7f2ac0158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e840 tx=0x7f2aac00ed00 comp rx=0 tx=0).stop 2024-04-26T22:26:50.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 shutdown_connections 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ac015d320 0x7f2ac0154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ac00d1090 0x7f2ac0158d10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.209+0000 7f2ac6ad1640 1 --2- 172.21.15.174:0/2463055107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac00cf990 0x7f2ac01f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.210+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 >> 172.21.15.174:0/2463055107 conn(0x7f2ac00d2f80 msgr2=0x7f2ac015d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.210+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 shutdown_connections 2024-04-26T22:26:50.211 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.210+0000 7f2ac6ad1640 1 -- 172.21.15.174:0/2463055107 wait complete. 2024-04-26T22:26:50.221 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new b9753598-6020-4660-980b-cbaf53601dac 1 2024-04-26T22:26:50.317 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.317+0000 7f3f1ea86640 1 Processor -- start 2024-04-26T22:26:50.317 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.317+0000 7f3f1ea86640 1 -- start start 2024-04-26T22:26:50.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1ea86640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3f180730d0 0x7f3f18073540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.319 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1ea86640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180d00a0 con 0x7f3f1815c0d0 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1ea86640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180d0380 con 0x7f3f180cfc10 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1ea86640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f3f18070500 con 0x7f3f180730d0 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f17fff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f3f180cfc10 0x7f3f180cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:43622/0 (socket says 172.21.15.174:43622) 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f17fff640 1 -- 172.21.15.174:0/615541804 learned_addr learned my addr 172.21.15.174:0/615541804 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1d023640 1 --2- 172.21.15.174:0/615541804 >> v2:172.21.15.62:3300/0 conn(0x7f3f180730d0 0x7f3f18073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202451754 0 0) 0x7f3f180d0380 con 0x7f3f180cfc10 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f08003520 con 0x7f3f180cfc10 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1d023640 1 -- 172.21.15.174:0/615541804 >> v1:172.21.15.174:6789/0 conn(0x7f3f180cfc10 legacy=0x7f3f180cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1d023640 1 -- 172.21.15.174:0/615541804 >> v1:172.21.15.62:6789/0 conn(0x7f3f1815c0d0 legacy=0x7f3f1806fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173924087 0 0) 0x7f3f180d00a0 con 0x7f3f1815c0d0 2024-04-26T22:26:50.320 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.318+0000 7f3f1d023640 1 -- 172.21.15.174:0/615541804 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180cd2e0 con 0x7f3f180730d0 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.319+0000 7f3f1d023640 1 --2- 172.21.15.174:0/615541804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180730d0 0x7f3f18073540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f18070680 tx=0x7f3f1002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06c95106facf85e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10007740 con 0x7f3f180730d0 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f100078e0 con 0x7f3f180730d0 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f177fe640 1 -- 172.21.15.174:0/615541804 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10007cb0 con 0x7f3f180730d0 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 -- 172.21.15.174:0/615541804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180730d0 msgr2=0x7f3f18073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/615541804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180730d0 0x7f3f18073540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f18070680 tx=0x7f3f1002a6f0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 -- 172.21.15.174:0/615541804 shutdown_connections 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/615541804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f180730d0 0x7f3f18073540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 -- 172.21.15.174:0/615541804 >> 172.21.15.174:0/615541804 conn(0x7f3f180d1e30 msgr2=0x7f3f180d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 -- 172.21.15.174:0/615541804 shutdown_connections 2024-04-26T22:26:50.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.320+0000 7f3f1ea86640 1 -- 172.21.15.174:0/615541804 wait complete. 2024-04-26T22:26:50.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.321+0000 7f3f1ea86640 1 Processor -- start 2024-04-26T22:26:50.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.321+0000 7f3f1ea86640 1 -- start start 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 0x7f3f18149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f1814a430 0x7f3f18156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815c4b0 con 0x7f3f1814a430 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180ce1f0 con 0x7f3f180730d0 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1ea86640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1814aa60 con 0x7f3f1815c0d0 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1c822640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1c822640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43686/0 (socket says 172.21.15.174:43686) 2024-04-26T22:26:50.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1c822640 1 -- 172.21.15.174:0/3965746761 learned_addr learned my addr 172.21.15.174:0/3965746761 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f17fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 0x7f3f18149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.322+0000 7f3f1d023640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f1814a430 0x7f3f18156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 -- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 msgr2=0x7f3f18149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 0x7f3f18149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 -- 172.21.15.174:0/3965746761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f1814a430 msgr2=0x7f3f18156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f1814a430 0x7f3f18156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 -- 172.21.15.174:0/3965746761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1814ad40 con 0x7f3f180730d0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f17fff640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 0x7f3f18149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1c822640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c460 tx=0x7f3f0c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c005480 con 0x7f3f180730d0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0c0035f0 con 0x7f3f180730d0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1814b020 con 0x7f3f180730d0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c003a00 con 0x7f3f180730d0 2024-04-26T22:26:50.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1814c250 con 0x7f3f180730d0 2024-04-26T22:26:50.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f0c003ba0 con 0x7f3f180730d0 2024-04-26T22:26:50.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.323+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3f0c012920 con 0x7f3f180730d0 2024-04-26T22:26:50.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.324+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f180693e0 con 0x7f3f180730d0 2024-04-26T22:26:50.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.327+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3f0c012b70 con 0x7f3f180730d0 2024-04-26T22:26:50.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.399+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b9753598-6020-4660-980b-cbaf53601dac", "id": 1} v 0) -- 0x7f3f18158260 con 0x7f3f180730d0 2024-04-26T22:26:50.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.404+0000 7f3f157fa640 1 -- 172.21.15.174:0/3965746761 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b9753598-6020-4660-980b-cbaf53601dac", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3f0c012e20 con 0x7f3f180730d0 2024-04-26T22:26:50.405 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-04-26T22:26:50.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 msgr2=0x7f3f18149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c460 tx=0x7f3f0c00c920 comp rx=0 tx=0).stop 2024-04-26T22:26:50.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 shutdown_connections 2024-04-26T22:26:50.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f1814a430 0x7f3f18156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f1815c0d0 0x7f3f18149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 --2- 172.21.15.174:0/3965746761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f180730d0 0x7f3f18149630 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 >> 172.21.15.174:0/3965746761 conn(0x7f3f180d1e30 msgr2=0x7f3f18158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 shutdown_connections 2024-04-26T22:26:50.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.405+0000 7f3f1ea86640 1 -- 172.21.15.174:0/3965746761 wait complete. 2024-04-26T22:26:50.418 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643 2 2024-04-26T22:26:50.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.515+0000 7f302a0b0640 1 Processor -- start 2024-04-26T22:26:50.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.515+0000 7f302a0b0640 1 -- start start 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f302a0b0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30240d0b00 0x7f30240ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f302a0b0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30240cf050 0x7f3024160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f302a0b0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240d1fb0 con 0x7f302415d400 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f302a0b0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f30240ce030 con 0x7f30240cf050 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f302a0b0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f30240ce310 con 0x7f30240d0b00 2024-04-26T22:26:50.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30240d0b00 0x7f30240ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f30237fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f302415d400 0x7f30240b2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.174:37120/0 (socket says 172.21.15.174:37120) 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30240d0b00 0x7f30240ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43702/0 (socket says 172.21.15.174:43702) 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f30237fe640 1 -- 172.21.15.174:0/431163031 learned_addr learned my addr 172.21.15.174:0/431163031 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f3023fff640 1 --2- 172.21.15.174:0/431163031 >> v2:172.21.15.62:3300/0 conn(0x7f30240cf050 0x7f3024160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f3022ffd640 1 -- 172.21.15.174:0/431163031 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992320549 0 0) 0x7f30240d1fb0 con 0x7f302415d400 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f3022ffd640 1 -- 172.21.15.174:0/431163031 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3010003520 con 0x7f302415d400 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 -- 172.21.15.174:0/431163031 >> v2:172.21.15.62:3300/0 conn(0x7f30240cf050 msgr2=0x7f3024160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 --2- 172.21.15.174:0/431163031 >> v2:172.21.15.62:3300/0 conn(0x7f30240cf050 0x7f3024160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 -- 172.21.15.174:0/431163031 >> v1:172.21.15.62:6789/0 conn(0x7f302415d400 legacy=0x7f30240b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.517+0000 7f301bfff640 1 -- 172.21.15.174:0/431163031 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240ce5f0 con 0x7f30240d0b00 2024-04-26T22:26:50.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.518+0000 7f301bfff640 1 --2- 172.21.15.174:0/431163031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30240d0b00 0x7f30240ceb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f302406fdc0 tx=0x7f300c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d39f392fa65f74e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.518+0000 7f3022ffd640 1 -- 172.21.15.174:0/431163031 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c037070 con 0x7f30240d0b00 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.518+0000 7f3022ffd640 1 -- 172.21.15.174:0/431163031 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c032440 con 0x7f30240d0b00 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.518+0000 7f3022ffd640 1 -- 172.21.15.174:0/431163031 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c032810 con 0x7f30240d0b00 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 -- 172.21.15.174:0/431163031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30240d0b00 msgr2=0x7f30240ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 --2- 172.21.15.174:0/431163031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30240d0b00 0x7f30240ceb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f302406fdc0 tx=0x7f300c02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 -- 172.21.15.174:0/431163031 shutdown_connections 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 --2- 172.21.15.174:0/431163031 >> v2:172.21.15.62:3300/0 conn(0x7f30240cf050 0x7f3024160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 --2- 172.21.15.174:0/431163031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30240d0b00 0x7f30240ceb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 -- 172.21.15.174:0/431163031 >> 172.21.15.174:0/431163031 conn(0x7f30240d2f30 msgr2=0x7f30240d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 -- 172.21.15.174:0/431163031 shutdown_connections 2024-04-26T22:26:50.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.519+0000 7f302a0b0640 1 -- 172.21.15.174:0/431163031 wait complete. 2024-04-26T22:26:50.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.521+0000 7f302a0b0640 1 Processor -- start 2024-04-26T22:26:50.522 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.521+0000 7f302a0b0640 1 -- start start 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.522+0000 7f302a0b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 0x7f30240cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.522+0000 7f302a0b0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 0x7f30241581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3024161af0 con 0x7f30240cf050 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30240cdd20 con 0x7f302415d400 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3024161c70 con 0x7f30240d0b00 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f301bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 0x7f30241581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43708/0 (socket says 172.21.15.174:43708) 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 -- 172.21.15.174:0/3105543659 learned_addr learned my addr 172.21.15.174:0/3105543659 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f30237fe640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 0x7f30240cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 -- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 msgr2=0x7f30241581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 0x7f30241581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 -- 172.21.15.174:0/3105543659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 msgr2=0x7f30240cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 0x7f30240cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 -- 172.21.15.174:0/3105543659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240cb990 con 0x7f302415d400 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f30237fe640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 0x7f30240cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f301bfff640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 0x7f30241581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3023fff640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f301c00efc0 tx=0x7f301c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c018020 con 0x7f302415d400 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301c014480 con 0x7f302415d400 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c014870 con 0x7f302415d400 2024-04-26T22:26:50.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30240cbc70 con 0x7f302415d400 2024-04-26T22:26:50.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30240cc230 con 0x7f302415d400 2024-04-26T22:26:50.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f301c005d10 con 0x7f302415d400 2024-04-26T22:26:50.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.523+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f301c013800 con 0x7f302415d400 2024-04-26T22:26:50.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.524+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30240d19d0 con 0x7f302415d400 2024-04-26T22:26:50.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.528+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f301c022070 con 0x7f302415d400 2024-04-26T22:26:50.603 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.603+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643", "id": 2} v 0) -- 0x7f30240693e0 con 0x7f302415d400 2024-04-26T22:26:50.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f3020ff9640 1 -- 172.21.15.174:0/3105543659 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f301c010040 con 0x7f302415d400 2024-04-26T22:26:50.609 INFO:teuthology.orchestra.run.smithi174.stdout:2 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 msgr2=0x7f302415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f301c00efc0 tx=0x7f301c00c490 comp rx=0 tx=0).stop 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 shutdown_connections 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302415d400 0x7f302415ead0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30240d0b00 0x7f30241581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 --2- 172.21.15.174:0/3105543659 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30240cf050 0x7f30240cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 >> 172.21.15.174:0/3105543659 conn(0x7f30240d2f30 msgr2=0x7f30240cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 shutdown_connections 2024-04-26T22:26:50.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.609+0000 7f302a0b0640 1 -- 172.21.15.174:0/3105543659 wait complete. 2024-04-26T22:26:50.620 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 4adc6b27-9944-45bb-8f9b-07c3b118cb8c 3 2024-04-26T22:26:50.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.718+0000 7fa726864640 1 Processor -- start 2024-04-26T22:26:50.719 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.718+0000 7fa726864640 1 -- start start 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa726864640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 0x7fa7200d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa726864640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa72015d870 con 0x7fa72015d490 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa726864640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7200c6d60 con 0x7fa7200d15a0 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa726864640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200d1f40 con 0x7fa7200c8b30 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa71f7fe640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fa7200d15a0 0x7fa7200b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:43628/0 (socket says 172.21.15.174:43628) 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa71f7fe640 1 -- 172.21.15.174:0/3413130539 learned_addr learned my addr 172.21.15.174:0/3413130539 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa724e01640 1 --2- 172.21.15.174:0/3413130539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 0x7fa7200d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911886248 0 0) 0x7fa7200c6d60 con 0x7fa7200d15a0 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa708003520 con 0x7fa7200d15a0 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815746208 0 0) 0x7fa72015d870 con 0x7fa72015d490 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa724e01640 1 -- 172.21.15.174:0/3413130539 >> v1:172.21.15.174:6789/0 conn(0x7fa7200d15a0 legacy=0x7fa7200b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa724e01640 1 -- 172.21.15.174:0/3413130539 >> v1:172.21.15.62:6789/0 conn(0x7fa72015d490 legacy=0x7fa7200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.722 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.719+0000 7fa724e01640 1 -- 172.21.15.174:0/3413130539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7200c7040 con 0x7fa7200c8b30 2024-04-26T22:26:50.722 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.720+0000 7fa724e01640 1 --2- 172.21.15.174:0/3413130539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 0x7fa7200d1a00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa718009dc0 tx=0x7fa71802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac6b0d349ec694a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.722 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.720+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718007740 con 0x7fa7200c8b30 2024-04-26T22:26:50.722 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.720+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7180078e0 con 0x7fa7200c8b30 2024-04-26T22:26:50.722 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.720+0000 7fa71effd640 1 -- 172.21.15.174:0/3413130539 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718007cb0 con 0x7fa7200c8b30 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 -- 172.21.15.174:0/3413130539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 msgr2=0x7fa7200d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 --2- 172.21.15.174:0/3413130539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 0x7fa7200d1a00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa718009dc0 tx=0x7fa71802a6d0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 -- 172.21.15.174:0/3413130539 shutdown_connections 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 --2- 172.21.15.174:0/3413130539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7200c8b30 0x7fa7200d1a00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 -- 172.21.15.174:0/3413130539 >> 172.21.15.174:0/3413130539 conn(0x7fa7200b4ab0 msgr2=0x7fa7200b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 -- 172.21.15.174:0/3413130539 shutdown_connections 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.721+0000 7fa726864640 1 -- 172.21.15.174:0/3413130539 wait complete. 2024-04-26T22:26:50.723 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.722+0000 7fa726864640 1 Processor -- start 2024-04-26T22:26:50.724 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.722+0000 7fa726864640 1 -- start start 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 0x7fa7200b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 0x7fa72015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200c7600 con 0x7fa7200c8b30 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200c6b10 con 0x7fa7200d15a0 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa726864640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200cecf0 con 0x7fa72015d490 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa71f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 0x7fa72015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.726 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57886/0 (socket says 172.21.15.174:57886) 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 -- 172.21.15.174:0/126162960 learned_addr learned my addr 172.21.15.174:0/126162960 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa71ffff640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 0x7fa7200b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 -- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 msgr2=0x7fa72015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 0x7fa72015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 -- 172.21.15.174:0/126162960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 msgr2=0x7fa7200b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 0x7fa7200b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa724e01640 1 -- 172.21.15.174:0/126162960 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7200cbec0 con 0x7fa72015d490 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa71f7fe640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 0x7fa72015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.723+0000 7fa71ffff640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 0x7fa7200b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa724e01640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa718009ee0 tx=0x7fa718004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718040070 con 0x7fa72015d490 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa718031bc0 con 0x7fa72015d490 2024-04-26T22:26:50.727 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72006d5a0 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718009450 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72006db60 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa718039040 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.724+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa718045db0 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.725+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa72006d120 con 0x7fa72015d490 2024-04-26T22:26:50.728 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.727+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa718053080 con 0x7fa72015d490 2024-04-26T22:26:50.801 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.801+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4adc6b27-9944-45bb-8f9b-07c3b118cb8c", "id": 3} v 0) -- 0x7fa7200693e0 con 0x7fa72015d490 2024-04-26T22:26:50.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.806+0000 7fa71cff9640 1 -- 172.21.15.174:0/126162960 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4adc6b27-9944-45bb-8f9b-07c3b118cb8c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa718007840 con 0x7fa72015d490 2024-04-26T22:26:50.807 INFO:teuthology.orchestra.run.smithi174.stdout:3 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 msgr2=0x7fa72015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa718009ee0 tx=0x7fa718004a40 comp rx=0 tx=0).stop 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 shutdown_connections 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa72015d490 0x7fa72015a570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7200d15a0 0x7fa72015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 --2- 172.21.15.174:0/126162960 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7200c8b30 0x7fa7200b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 >> 172.21.15.174:0/126162960 conn(0x7fa7200b4ab0 msgr2=0x7fa7200cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 shutdown_connections 2024-04-26T22:26:50.808 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.807+0000 7fa726864640 1 -- 172.21.15.174:0/126162960 wait complete. 2024-04-26T22:26:50.818 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 6e454f90-4f3f-48f4-a699-9d309757b2d8 4 2024-04-26T22:26:50.878 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:50.877+0000 7f9d3d61e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:26:50.890 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:50.888+0000 7f5eb4012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T22:26:50.915 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.915+0000 7f8d6099d640 1 Processor -- start 2024-04-26T22:26:50.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.915+0000 7f8d6099d640 1 -- start start 2024-04-26T22:26:50.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8d5c15d320 0x7f8d5c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 0x7f8d5c0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c0ceeb0 con 0x7f8d5c0cf990 2024-04-26T22:26:50.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8d5c15d700 con 0x7f8d5c15d320 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d6099d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c0ce8f0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8d5c15d320 0x7f8d5c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d59d9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8d5c15d320 0x7f8d5c0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43734/0 (socket says 172.21.15.174:43734) 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d59d9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57894/0 (socket says 172.21.15.174:57894) 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 -- 172.21.15.174:0/739910877 learned_addr learned my addr 172.21.15.174:0/739910877 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5ad9d640 1 --2- 172.21.15.174:0/739910877 >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 0x7f8d5c0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 -- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 msgr2=0x7f8d5c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 -- 172.21.15.174:0/739910877 >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 msgr2=0x7f8d5c0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 --2- 172.21.15.174:0/739910877 >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 0x7f8d5c0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5a59c640 1 -- 172.21.15.174:0/739910877 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0cebd0 con 0x7f8d5c15d320 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d59d9b640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:50.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.916+0000 7f8d5ad9d640 1 --2- 172.21.15.174:0/739910877 >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 0x7f8d5c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.917+0000 7f8d5a59c640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c15d320 0x7f8d5c0d0b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06cd50 tx=0x7f8d4c02a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c42cc2059c50821 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.917+0000 7f8d5959a640 1 -- 172.21.15.174:0/739910877 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c037070 con 0x7f8d5c15d320 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.917+0000 7f8d5959a640 1 -- 172.21.15.174:0/739910877 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4c032440 con 0x7f8d5c15d320 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.917+0000 7f8d5959a640 1 -- 172.21.15.174:0/739910877 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c032810 con 0x7f8d5c15d320 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 -- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c15d320 msgr2=0x7f8d5c0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c15d320 0x7f8d5c0d0b50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06cd50 tx=0x7f8d4c02a140 comp rx=0 tx=0).stop 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 -- 172.21.15.174:0/739910877 shutdown_connections 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 --2- 172.21.15.174:0/739910877 >> v2:172.21.15.62:3300/0 conn(0x7f8d5c0cf990 0x7f8d5c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0d1090 0x7f8d5c0d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 --2- 172.21.15.174:0/739910877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c15d320 0x7f8d5c0d0b50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 -- 172.21.15.174:0/739910877 >> 172.21.15.174:0/739910877 conn(0x7f8d5c0d2f80 msgr2=0x7f8d5c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 -- 172.21.15.174:0/739910877 shutdown_connections 2024-04-26T22:26:50.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.918+0000 7f8d6099d640 1 -- 172.21.15.174:0/739910877 wait complete. 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.919+0000 7f8d6099d640 1 Processor -- start 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.919+0000 7f8d6099d640 1 -- start start 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0cf990 0x7f8d5c0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c15d320 0x7f8d5c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0ce550 con 0x7f8d5c15d320 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c0b2cd0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c0ce6d0 con 0x7f8d5c0cf990 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d5a59c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0cf990 0x7f8d5c0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43744/0 (socket says 172.21.15.174:43744) 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 -- 172.21.15.174:0/1692915945 learned_addr learned my addr 172.21.15.174:0/1692915945 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d5ad9d640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c15d320 0x7f8d5c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:50.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 -- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0cf990 msgr2=0x7f8d5c0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0cf990 0x7f8d5c0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 -- 172.21.15.174:0/1692915945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c15d320 msgr2=0x7f8d5c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c15d320 0x7f8d5c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 -- 172.21.15.174:0/1692915945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c158710 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d59d9b640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e960 tx=0x7f8d5000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d50017070 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d5000ca00 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c1f9fa0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.920+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5000cdd0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.921+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c158990 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.921+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d5001a070 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.921+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8d50013470 con 0x7f8d5c0d1090 2024-04-26T22:26:50.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.921+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c069bb0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.925 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.925+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8d5c069bb0 con 0x7f8d5c0d1090 2024-04-26T22:26:50.970 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:50.970+0000 7f9d3d61e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:26:50.982 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:50.981+0000 7f5eb4012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T22:26:50.998 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:50.998+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e454f90-4f3f-48f4-a699-9d309757b2d8", "id": 4} v 0) -- 0x7f8d5c0693e0 con 0x7f8d5c0d1090 2024-04-26T22:26:51.004 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.003+0000 7f8d4affd640 1 -- 172.21.15.174:0/1692915945 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e454f90-4f3f-48f4-a699-9d309757b2d8", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8d5c1f9fa0 con 0x7f8d5c0d1090 2024-04-26T22:26:51.004 INFO:teuthology.orchestra.run.smithi174.stdout:4 2024-04-26T22:26:51.005 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 msgr2=0x7f8d5c1f9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.005 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d5000e960 tx=0x7f8d5000ee20 comp rx=0 tx=0).stop 2024-04-26T22:26:51.005 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 shutdown_connections 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8d5c15d320 0x7f8d5c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d5c0d1090 0x7f8d5c1f9a60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 --2- 172.21.15.174:0/1692915945 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8d5c0cf990 0x7f8d5c0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 >> 172.21.15.174:0/1692915945 conn(0x7f8d5c0d2f80 msgr2=0x7f8d5c0cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 shutdown_connections 2024-04-26T22:26:51.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.004+0000 7f8d6099d640 1 -- 172.21.15.174:0/1692915945 wait complete. 2024-04-26T22:26:51.015 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new bbd7a770-7a75-4e2d-a413-2bff16d3bd05 5 2024-04-26T22:26:51.016 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.016+0000 7f9d3d61e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:26:51.027 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.026+0000 7f5eb4012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T22:26:51.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.112+0000 7f219a6d9640 1 Processor -- start 2024-04-26T22:26:51.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.112+0000 7f219a6d9640 1 -- start start 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f219a6d9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f219415f560 0x7f21940d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f219a6d9640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21940d23a0 0x7f21941693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f219a6d9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219415f940 con 0x7f21940d10a0 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f219a6d9640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f21940d43a0 con 0x7f219415f560 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f219a6d9640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f21940d3300 con 0x7f21940d23a0 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21940d23a0 0x7f21941693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21940d23a0 0x7f21941693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43758/0 (socket says 172.21.15.174:43758) 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 -- 172.21.15.174:0/2809237164 learned_addr learned my addr 172.21.15.174:0/2809237164 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2193fff640 1 --2- 172.21.15.174:0/2809237164 >> v2:172.21.15.62:3300/0 conn(0x7f219415f560 0x7f21940d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 -- 172.21.15.174:0/2809237164 >> v2:172.21.15.62:3300/0 conn(0x7f219415f560 msgr2=0x7f21940d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 --2- 172.21.15.174:0/2809237164 >> v2:172.21.15.62:3300/0 conn(0x7f219415f560 0x7f21940d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 -- 172.21.15.174:0/2809237164 >> v1:172.21.15.62:6789/0 conn(0x7f21940d10a0 legacy=0x7f21940d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.113+0000 7f2198c76640 1 -- 172.21.15.174:0/2809237164 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940d35e0 con 0x7f21940d23a0 2024-04-26T22:26:51.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.114+0000 7f2198c76640 1 --2- 172.21.15.174:0/2809237164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f21941693c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21940cf8c0 tx=0x7f218c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea16c9a1d3c7af6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.114+0000 7f2192ffd640 1 -- 172.21.15.174:0/2809237164 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c036070 con 0x7f21940d23a0 2024-04-26T22:26:51.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.114+0000 7f2192ffd640 1 -- 172.21.15.174:0/2809237164 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c031600 con 0x7f21940d23a0 2024-04-26T22:26:51.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.114+0000 7f2192ffd640 1 -- 172.21.15.174:0/2809237164 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218c0319d0 con 0x7f21940d23a0 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 -- 172.21.15.174:0/2809237164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 msgr2=0x7f21941693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 --2- 172.21.15.174:0/2809237164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f21941693c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f21940cf8c0 tx=0x7f218c02a730 comp rx=0 tx=0).stop 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 -- 172.21.15.174:0/2809237164 shutdown_connections 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 --2- 172.21.15.174:0/2809237164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f21941693c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 --2- 172.21.15.174:0/2809237164 >> v2:172.21.15.62:3300/0 conn(0x7f219415f560 0x7f21940d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 -- 172.21.15.174:0/2809237164 >> 172.21.15.174:0/2809237164 conn(0x7f219409e570 msgr2=0x7f21940d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 -- 172.21.15.174:0/2809237164 shutdown_connections 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.115+0000 7f219a6d9640 1 -- 172.21.15.174:0/2809237164 wait complete. 2024-04-26T22:26:51.117 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.116+0000 7f219a6d9640 1 Processor -- start 2024-04-26T22:26:51.118 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.116+0000 7f219a6d9640 1 -- start start 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 0x7f21940b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 0x7f2194161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21940d3e80 con 0x7f21940d10a0 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21940d3ba0 con 0x7f21940d23a0 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f219a6d9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f21940b6ac0 con 0x7f219415f560 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f2198c76640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 0x7f2194161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f2198c76640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 0x7f2194161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57912/0 (socket says 172.21.15.174:57912) 2024-04-26T22:26:51.120 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43764/0 (socket says 172.21.15.174:43764) 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f2198c76640 1 -- 172.21.15.174:0/3015196132 learned_addr learned my addr 172.21.15.174:0/3015196132 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f2193fff640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 0x7f21940b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 -- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 msgr2=0x7f2194161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 0x7f2194161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 -- 172.21.15.174:0/3015196132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 msgr2=0x7f21940b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 0x7f21940b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f21937fe640 1 -- 172.21.15.174:0/3015196132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940ce650 con 0x7f21940d23a0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.117+0000 7f2193fff640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 0x7f21940b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f21937fe640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f218400c3c0 tx=0x7f218400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21840052d0 con 0x7f21940d23a0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2184005490 con 0x7f21940d23a0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184007260 con 0x7f21940d23a0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21940ce930 con 0x7f21940d23a0 2024-04-26T22:26:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21940ceef0 con 0x7f21940d23a0 2024-04-26T22:26:51.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2184007400 con 0x7f21940d23a0 2024-04-26T22:26:51.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.118+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f21840035f0 con 0x7f21940d23a0 2024-04-26T22:26:51.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.119+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219406a360 con 0x7f21940d23a0 2024-04-26T22:26:51.122 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.121+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2184003aa0 con 0x7f21940d23a0 2024-04-26T22:26:51.191 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.191+0000 7f9d3d61e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:26:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.193+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bbd7a770-7a75-4e2d-a413-2bff16d3bd05", "id": 5} v 0) -- 0x7f21940d2a40 con 0x7f21940d23a0 2024-04-26T22:26:51.197 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.196+0000 7f2190ff9640 1 -- 172.21.15.174:0/3015196132 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bbd7a770-7a75-4e2d-a413-2bff16d3bd05", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2184003c80 con 0x7f21940d23a0 2024-04-26T22:26:51.197 INFO:teuthology.orchestra.run.smithi174.stdout:5 2024-04-26T22:26:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 msgr2=0x7f2194161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f218400c3c0 tx=0x7f218400c880 comp rx=0 tx=0).stop 2024-04-26T22:26:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 shutdown_connections 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f219415f560 0x7f2194161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21940d23a0 0x7f2194161020 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 --2- 172.21.15.174:0/3015196132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21940d10a0 0x7f21940b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.197+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 >> 172.21.15.174:0/3015196132 conn(0x7f219409e570 msgr2=0x7f21940b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.198+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 shutdown_connections 2024-04-26T22:26:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.198+0000 7f219a6d9640 1 -- 172.21.15.174:0/3015196132 wait complete. 2024-04-26T22:26:51.202 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.201+0000 7f5eb4012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T22:26:51.208 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new dbea0fd7-32c3-4a0c-9963-25e5da3f32da 6 2024-04-26T22:26:51.246 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.245+0000 7f9d3d61e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:26:51.258 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.257+0000 7f5eb4012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T22:26:51.300 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.299+0000 7f9d3d61e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:26:51.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.304+0000 7f85cd99d640 1 Processor -- start 2024-04-26T22:26:51.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.304+0000 7f85cd99d640 1 -- start start 2024-04-26T22:26:51.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85cd99d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85c806c950 0x7f85c8148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85cd99d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 0x7f85c80d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85cd99d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85c8145d70 con 0x7f85c8145990 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85cd99d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f85c80703a0 con 0x7f85c806c950 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85cd99d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f85c80cbcc0 con 0x7f85c806f3e0 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c6ffd640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f85c8145990 0x7f85c806c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:43632/0 (socket says 172.21.15.174:43632) 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 0x7f85c80d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c6ffd640 1 -- 172.21.15.174:0/3748472462 learned_addr learned my addr 172.21.15.174:0/3748472462 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c67fc640 1 --2- 172.21.15.174:0/3748472462 >> v2:172.21.15.62:3300/0 conn(0x7f85c806c950 0x7f85c8148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c5ffb640 1 -- 172.21.15.174:0/3748472462 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182429658 0 0) 0x7f85c8145d70 con 0x7f85c8145990 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c5ffb640 1 -- 172.21.15.174:0/3748472462 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85b0003520 con 0x7f85c8145990 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c77fe640 1 -- 172.21.15.174:0/3748472462 >> v2:172.21.15.62:3300/0 conn(0x7f85c806c950 msgr2=0x7f85c8148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c77fe640 1 --2- 172.21.15.174:0/3748472462 >> v2:172.21.15.62:3300/0 conn(0x7f85c806c950 0x7f85c8148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c77fe640 1 -- 172.21.15.174:0/3748472462 >> v1:172.21.15.174:6789/0 conn(0x7f85c8145990 legacy=0x7f85c806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.305+0000 7f85c77fe640 1 -- 172.21.15.174:0/3748472462 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c80cbfa0 con 0x7f85c806f3e0 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.306+0000 7f85c77fe640 1 --2- 172.21.15.174:0/3748472462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 0x7f85c80d0680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009d90 tx=0x7f85c002a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12446266461fbce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85c5ffb640 1 -- 172.21.15.174:0/3748472462 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0007740 con 0x7f85c806f3e0 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85c5ffb640 1 -- 172.21.15.174:0/3748472462 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c00078e0 con 0x7f85c806f3e0 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85c5ffb640 1 -- 172.21.15.174:0/3748472462 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0007cb0 con 0x7f85c806f3e0 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 -- 172.21.15.174:0/3748472462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 msgr2=0x7f85c80d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 --2- 172.21.15.174:0/3748472462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 0x7f85c80d0680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009d90 tx=0x7f85c002a530 comp rx=0 tx=0).stop 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 -- 172.21.15.174:0/3748472462 shutdown_connections 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 --2- 172.21.15.174:0/3748472462 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806f3e0 0x7f85c80d0680 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 --2- 172.21.15.174:0/3748472462 >> v2:172.21.15.62:3300/0 conn(0x7f85c806c950 0x7f85c8148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 -- 172.21.15.174:0/3748472462 >> 172.21.15.174:0/3748472462 conn(0x7f85c80d1e30 msgr2=0x7f85c80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 -- 172.21.15.174:0/3748472462 shutdown_connections 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.307+0000 7f85cd99d640 1 -- 172.21.15.174:0/3748472462 wait complete. 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.308+0000 7f85cd99d640 1 Processor -- start 2024-04-26T22:26:51.310 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 -- start start 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806f3e0 0x7f85c8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 0x7f85c8147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8148f60 con 0x7f85c806f3e0 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8070030 con 0x7f85c8145990 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85cd99d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f85c80d0bc0 con 0x7f85c806c950 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57918/0 (socket says 172.21.15.174:57918) 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 0x7f85c8147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.313 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85c6ffd640 1 -- 172.21.15.174:0/117337688 learned_addr learned my addr 172.21.15.174:0/117337688 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.309+0000 7f85c67fc640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806f3e0 0x7f85c8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 -- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 msgr2=0x7f85c8147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 0x7f85c8147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 -- 172.21.15.174:0/117337688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806f3e0 msgr2=0x7f85c8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806f3e0 0x7f85c8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 -- 172.21.15.174:0/117337688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c8155430 con 0x7f85c806c950 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c77fe640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 0x7f85c8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85c6ffd640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c3c0 tx=0x7f85bc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc017070 con 0x7f85c806c950 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85bc005ce0 con 0x7f85c806c950 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0072f0 con 0x7f85c806c950 2024-04-26T22:26:51.314 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c8155710 con 0x7f85c806c950 2024-04-26T22:26:51.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c8152b40 con 0x7f85c806c950 2024-04-26T22:26:51.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85bc01a070 con 0x7f85c806c950 2024-04-26T22:26:51.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.310+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f85bc0129f0 con 0x7f85c806c950 2024-04-26T22:26:51.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.311+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c80693e0 con 0x7f85c806c950 2024-04-26T22:26:51.315 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.313+0000 7f5eb4012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T22:26:51.315 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.313+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f85bc021020 con 0x7f85c806c950 2024-04-26T22:26:51.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.384+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dbea0fd7-32c3-4a0c-9963-25e5da3f32da", "id": 6} v 0) -- 0x7f85c8073c10 con 0x7f85c806c950 2024-04-26T22:26:51.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.390+0000 7f85affff640 1 -- 172.21.15.174:0/117337688 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbea0fd7-32c3-4a0c-9963-25e5da3f32da", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f85bc007600 con 0x7f85c806c950 2024-04-26T22:26:51.390 INFO:teuthology.orchestra.run.smithi174.stdout:6 2024-04-26T22:26:51.391 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.390+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 msgr2=0x7f85c80ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c3c0 tx=0x7f85bc00c880 comp rx=0 tx=0).stop 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 shutdown_connections 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85c8145990 0x7f85c8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85c806f3e0 0x7f85c8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 --2- 172.21.15.174:0/117337688 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f85c806c950 0x7f85c80ae2e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 >> 172.21.15.174:0/117337688 conn(0x7f85c80d1e30 msgr2=0x7f85c80cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 shutdown_connections 2024-04-26T22:26:51.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.391+0000 7f85cd99d640 1 -- 172.21.15.174:0/117337688 wait complete. 2024-04-26T22:26:51.402 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new b657c762-6714-4b75-bc73-5f19180dd4b8 7 2024-04-26T22:26:51.416 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.416+0000 7f9d3d61e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:26:51.432 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.432+0000 7f5eb4012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T22:26:51.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.512+0000 7f0ea2cda640 1 Processor -- start 2024-04-26T22:26:51.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.512+0000 7f0ea2cda640 1 -- start start 2024-04-26T22:26:51.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea2cda640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0e9c15c470 0x7f0e9c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea2cda640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 0x7f0e9c15fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea2cda640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e9c0cccc0 con 0x7f0e9c15c090 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea2cda640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0e9c0d11e0 con 0x7f0e9c15c470 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea2cda640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c0cd590 con 0x7f0e9c0d09a0 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0e9c15c470 0x7f0e9c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0e9c15c470 0x7f0e9c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43792/0 (socket says 172.21.15.174:43792) 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea1277640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 0x7f0e9c15fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3919309129 learned_addr learned my addr 172.21.15.174:0/3919309129 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 msgr2=0x7f0e9c15fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 0x7f0e9c15fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3919309129 >> v1:172.21.15.62:6789/0 conn(0x7f0e9c15c090 legacy=0x7f0e9c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3919309129 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0d0f00 con 0x7f0e9c15c470 2024-04-26T22:26:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.514+0000 7f0ea1277640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 0x7f0e9c15fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.515+0000 7f0e9bfff640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c15c470 0x7f0e9c0d0460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009870 tx=0x7f0e8802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf2fa7a1c691bfd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.515+0000 7f0e9b7fe640 1 -- 172.21.15.174:0/3919309129 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88037070 con 0x7f0e9c15c470 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.515+0000 7f0e9b7fe640 1 -- 172.21.15.174:0/3919309129 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8802ae50 con 0x7f0e9c15c470 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.515+0000 7f0e9b7fe640 1 -- 172.21.15.174:0/3919309129 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88032860 con 0x7f0e9c15c470 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c15c470 msgr2=0x7f0e9c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c15c470 0x7f0e9c0d0460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009870 tx=0x7f0e8802a300 comp rx=0 tx=0).stop 2024-04-26T22:26:51.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3919309129 shutdown_connections 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c15c470 0x7f0e9c0d0460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3919309129 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c0d09a0 0x7f0e9c15fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3919309129 >> 172.21.15.174:0/3919309129 conn(0x7f0e9c0d1e80 msgr2=0x7f0e9c0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3919309129 shutdown_connections 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.516+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3919309129 wait complete. 2024-04-26T22:26:51.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.518+0000 7f0ea2cda640 1 Processor -- start 2024-04-26T22:26:51.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.518+0000 7f0ea2cda640 1 -- start start 2024-04-26T22:26:51.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e9c0c9ea0 0x7f0e9c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c160b20 con 0x7f0e9c0c9ea0 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0cd220 con 0x7f0e9c0d09a0 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea2cda640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c1602a0 con 0x7f0e9c15c090 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea0a76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:57942/0 (socket says 172.21.15.174:57942) 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea0a76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43804/0 (socket says 172.21.15.174:43804) 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3926498443 learned_addr learned my addr 172.21.15.174:0/3926498443 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea1277640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e9c0c9ea0 0x7f0e9c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 msgr2=0x7f0e9c15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3926498443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e9c0c9ea0 msgr2=0x7f0e9c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e9c0c9ea0 0x7f0e9c0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 -- 172.21.15.174:0/3926498443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c0ca3b0 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0ea0a76640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e9bfff640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e8802a340 tx=0x7f0e880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.520+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88041070 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8803c680 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e88037040 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c0ca690 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c15db90 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e88044070 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.521+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0e8803d3f0 con 0x7f0e9c15c090 2024-04-26T22:26:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.522+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e9c076910 con 0x7f0e9c15c090 2024-04-26T22:26:51.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.524+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0e9c076910 con 0x7f0e9c15c090 2024-04-26T22:26:51.597 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.596+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b657c762-6714-4b75-bc73-5f19180dd4b8", "id": 7} v 0) -- 0x7f0e9c0b7e00 con 0x7f0e9c15c090 2024-04-26T22:26:51.602 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.602+0000 7f0e997fa640 1 -- 172.21.15.174:0/3926498443 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b657c762-6714-4b75-bc73-5f19180dd4b8", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0e88046450 con 0x7f0e9c15c090 2024-04-26T22:26:51.603 INFO:teuthology.orchestra.run.smithi174.stdout:7 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 msgr2=0x7f0e9c0c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e8802a340 tx=0x7f0e880046c0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 shutdown_connections 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e9c0c9ea0 0x7f0e9c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e9c15c090 0x7f0e9c0c9960 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 --2- 172.21.15.174:0/3926498443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c0d09a0 0x7f0e9c15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 >> 172.21.15.174:0/3926498443 conn(0x7f0e9c0d1e80 msgr2=0x7f0e9c06ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 shutdown_connections 2024-04-26T22:26:51.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:26:51.603+0000 7f0ea2cda640 1 -- 172.21.15.174:0/3926498443 wait complete. 2024-04-26T22:26:51.614 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T22:26:51.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T22:26:51.616 INFO:tasks.ceph.osd.0:Started 2024-04-26T22:26:51.616 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T22:26:51.617 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T22:26:51.620 INFO:tasks.ceph.osd.1:Started 2024-04-26T22:26:51.620 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T22:26:51.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T22:26:51.622 INFO:tasks.ceph.osd.2:Started 2024-04-26T22:26:51.623 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T22:26:51.623 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T22:26:51.625 INFO:tasks.ceph.osd.3:Started 2024-04-26T22:26:51.625 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T22:26:51.625 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T22:26:51.627 INFO:tasks.ceph.osd.4:Started 2024-04-26T22:26:51.627 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T22:26:51.627 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T22:26:51.629 INFO:tasks.ceph.osd.5:Started 2024-04-26T22:26:51.629 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T22:26:51.629 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T22:26:51.631 INFO:tasks.ceph.osd.6:Started 2024-04-26T22:26:51.631 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T22:26:51.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T22:26:51.633 INFO:tasks.ceph.osd.7:Started 2024-04-26T22:26:51.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:26:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.769+0000 7f4a2ca83640 1 Processor -- start 2024-04-26T22:26:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.769+0000 7f4a2ca83640 1 -- start start 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2ca83640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4a2815f400 0x7f4a280d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2ca83640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a280d3e50 0x7f4a280b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2ca83640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a280d3220 con 0x7f4a280d0b80 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2ca83640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f4a280d26c0 con 0x7f4a2815f400 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2ca83640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4a280d5ef0 con 0x7f4a280d3e50 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4a2815f400 0x7f4a280d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4a2815f400 0x7f4a280d2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39768/0 (socket says 172.21.15.62:39768) 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 -- 172.21.15.62:0/1039249700 learned_addr learned my addr 172.21.15.62:0/1039249700 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a26d9d640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a280d3e50 0x7f4a280b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 -- 172.21.15.62:0/1039249700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a280d3e50 msgr2=0x7f4a280b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a280d3e50 0x7f4a280b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 -- 172.21.15.62:0/1039249700 >> v1:172.21.15.174:6789/0 conn(0x7f4a280d0b80 legacy=0x7f4a280d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.770+0000 7f4a2659c640 1 -- 172.21.15.62:0/1039249700 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280d00a0 con 0x7f4a2815f400 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.771+0000 7f4a2659c640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a2815f400 0x7f4a280d2180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009870 tx=0x7f4a1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff2494a378bb80d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.771+0000 7f4a2559a640 1 -- 172.21.15.62:0/1039249700 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c037070 con 0x7f4a2815f400 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.771+0000 7f4a2559a640 1 -- 172.21.15.62:0/1039249700 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a1c032440 con 0x7f4a2815f400 2024-04-26T22:26:51.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.771+0000 7f4a2559a640 1 -- 172.21.15.62:0/1039249700 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c032810 con 0x7f4a2815f400 2024-04-26T22:26:51.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 -- 172.21.15.62:0/1039249700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a2815f400 msgr2=0x7f4a280d2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a2815f400 0x7f4a280d2180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009870 tx=0x7f4a1c02a270 comp rx=0 tx=0).stop 2024-04-26T22:26:51.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 -- 172.21.15.62:0/1039249700 shutdown_connections 2024-04-26T22:26:51.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a280d3e50 0x7f4a280b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/1039249700 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a2815f400 0x7f4a280d2180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 -- 172.21.15.62:0/1039249700 >> 172.21.15.62:0/1039249700 conn(0x7f4a2809e510 msgr2=0x7f4a280d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 -- 172.21.15.62:0/1039249700 shutdown_connections 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.772+0000 7f4a2ca83640 1 -- 172.21.15.62:0/1039249700 wait complete. 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.773+0000 7f4a2ca83640 1 Processor -- start 2024-04-26T22:26:51.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.773+0000 7f4a2ca83640 1 -- start start 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 0x7f4a280cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a25d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a25d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:39770/0 (socket says 172.21.15.62:39770) 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a2815f400 0x7f4a28152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 0x7f4a280cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 0x7f4a280cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:58434/0 (socket says 172.21.15.62:58434) 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a25d9b640 1 -- 172.21.15.62:0/2562872720 learned_addr learned my addr 172.21.15.62:0/2562872720 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4a280cfd50 con 0x7f4a280d3e50 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a280d0660 con 0x7f4a280d0b80 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2815f7e0 con 0x7f4a2815f400 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.774+0000 7f4a26d9d640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a2815f400 0x7f4a28152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 -- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a2815f400 msgr2=0x7f4a28152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a2815f400 0x7f4a28152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 -- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 msgr2=0x7f4a280cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 0x7f4a280cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280cf3f0 con 0x7f4a280d3e50 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a25d9b640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c400 tx=0x7f4a1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18017070 con 0x7f4a280d3e50 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a180052d0 con 0x7f4a280d3e50 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a280b8f70 con 0x7f4a280d3e50 2024-04-26T22:26:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a18003830 con 0x7f4a280d3e50 2024-04-26T22:26:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a280cf6d0 con 0x7f4a280d3e50 2024-04-26T22:26:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a1801a070 con 0x7f4a280d3e50 2024-04-26T22:26:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.775+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f4a18011590 con 0x7f4a280d3e50 2024-04-26T22:26:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.776+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a280d2410 con 0x7f4a280d3e50 2024-04-26T22:26:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.778+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4a280d2410 con 0x7f4a280d3e50 2024-04-26T22:26:51.831 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:51.831+0000 7f9d3d61e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:26:51.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.841+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a280d2880 con 0x7f4a280d3e50 2024-04-26T22:26:51.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.842+0000 7f4a16ffd640 1 -- 172.21.15.62:0/2562872720 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f4a18021020 con 0x7f4a280d3e50 2024-04-26T22:26:51.842 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:26:51.842 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":10,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:26:51.598583+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 msgr2=0x7f4a280ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c400 tx=0x7f4a1800c8c0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 shutdown_connections 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4a2815f400 0x7f4a28152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4a280d3e50 0x7f4a280ceb50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 --2- 172.21.15.62:0/2562872720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a280d0b80 0x7f4a280cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:51.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 >> 172.21.15.62:0/2562872720 conn(0x7f4a2809e510 msgr2=0x7f4a280d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 shutdown_connections 2024-04-26T22:26:51.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:26:51.843+0000 7f4a2ca83640 1 -- 172.21.15.62:0/2562872720 wait complete. 2024-04-26T22:26:51.854 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T22:26:51.854 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T22:26:51.874 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:51.874+0000 7f5eb4012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T22:26:52.087 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.086+0000 7f9d3d61e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:26:52.138 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.138+0000 7f9d3d61e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:26:52.146 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.146+0000 7f5eb4012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T22:26:52.200 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.200+0000 7f5eb4012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T22:26:52.225 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-26T22:26:52.225+0000 7f7323232640 -1 Falling back to public interface 2024-04-26T22:26:52.238 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-04-26T22:26:52.237+0000 7f6da72af640 -1 Falling back to public interface 2024-04-26T22:26:52.241 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-04-26T22:26:52.241+0000 7f6b0fa2d640 -1 Falling back to public interface 2024-04-26T22:26:52.245 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-26T22:26:52.245+0000 7f31fdaea640 -1 Falling back to public interface 2024-04-26T22:26:52.246 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-04-26T22:26:52.245+0000 7f5e3f5ba640 -1 Falling back to public interface 2024-04-26T22:26:52.246 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-26T22:26:52.247+0000 7f807872c640 -1 Falling back to public interface 2024-04-26T22:26:52.248 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-04-26T22:26:52.248+0000 7f51adc86640 -1 Falling back to public interface 2024-04-26T22:26:52.250 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-26T22:26:52.250+0000 7f32275a1640 -1 Falling back to public interface 2024-04-26T22:26:52.259 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.259+0000 7f9d3d61e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:26:52.328 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.328+0000 7f5eb4012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T22:26:52.538 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.535+0000 7f9d3d61e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:26:52.591 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.583+0000 7f9d3d61e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:26:52.625 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.624+0000 7f5eb4012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T22:26:52.660 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T22:26:52.674 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.675+0000 7f5eb4012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T22:26:52.738 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.737+0000 7f9d3d61e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:26:52.753 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.753+0000 7f9071904640 1 Processor -- start 2024-04-26T22:26:52.753 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.753+0000 7f9071904640 1 -- start start 2024-04-26T22:26:52.754 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f9071904640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f906c0d3a10 0x7f906c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f9071904640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f906c0b81b0 con 0x7f906c0d3df0 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f9071904640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f906c0b8490 con 0x7f906c15f800 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f9071904640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f906c0b8770 con 0x7f906c0d3a10 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f906c0d3a10 0x7f906c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f906c0d3a10 0x7f906c0d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55148/0 (socket says 172.21.15.62:55148) 2024-04-26T22:26:52.755 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906a7fc640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f906c0d3df0 0x7f906c15f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60656/0 (socket says 172.21.15.62:60656) 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 -- 172.21.15.62:0/1629717346 learned_addr learned my addr 172.21.15.62:0/1629717346 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 -- 172.21.15.62:0/1629717346 >> v1:172.21.15.174:6789/0 conn(0x7f906c15f800 legacy=0x7f906c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 -- 172.21.15.62:0/1629717346 >> v1:172.21.15.62:6789/0 conn(0x7f906c0d3df0 legacy=0x7f906c15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f9069ffb640 1 -- 172.21.15.62:0/1629717346 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3691233533 0 0) 0x7f906c0b81b0 con 0x7f906c0d3df0 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.754+0000 7f906affd640 1 -- 172.21.15.62:0/1629717346 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c0b8a50 con 0x7f906c0d3a10 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.755+0000 7f906affd640 1 --2- 172.21.15.62:0/1629717346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3a10 0x7f906c0d0890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2519227499aecf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.755+0000 7f9069ffb640 1 -- 172.21.15.62:0/1629717346 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c037070 con 0x7f906c0d3a10 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.755+0000 7f9069ffb640 1 -- 172.21.15.62:0/1629717346 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c032440 con 0x7f906c0d3a10 2024-04-26T22:26:52.756 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.755+0000 7f9069ffb640 1 -- 172.21.15.62:0/1629717346 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c032810 con 0x7f906c0d3a10 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 -- 172.21.15.62:0/1629717346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3a10 msgr2=0x7f906c0d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 --2- 172.21.15.62:0/1629717346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3a10 0x7f906c0d0890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c02a270 comp rx=0 tx=0).stop 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 -- 172.21.15.62:0/1629717346 shutdown_connections 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 --2- 172.21.15.62:0/1629717346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3a10 0x7f906c0d0890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 -- 172.21.15.62:0/1629717346 >> 172.21.15.62:0/1629717346 conn(0x7f906c09e520 msgr2=0x7f906c0d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:52.757 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 -- 172.21.15.62:0/1629717346 shutdown_connections 2024-04-26T22:26:52.758 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.756+0000 7f9071904640 1 -- 172.21.15.62:0/1629717346 wait complete. 2024-04-26T22:26:52.758 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.757+0000 7f9071904640 1 Processor -- start 2024-04-26T22:26:52.758 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.757+0000 7f9071904640 1 -- start start 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f906c0d3a10 0x7f906c1f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906c15f800 0x7f906c09ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0ba3e0 con 0x7f906c0d3df0 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0ba260 con 0x7f906c15f800 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f9071904640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f906c0ba560 con 0x7f906c0d3a10 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55164/0 (socket says 172.21.15.62:55164) 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 -- 172.21.15.62:0/4165102620 learned_addr learned my addr 172.21.15.62:0/4165102620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906b7fe640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906c15f800 0x7f906c09ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906affd640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f906c0d3a10 0x7f906c1f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:52.760 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 -- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f906c0d3a10 msgr2=0x7f906c1f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f906c0d3a10 0x7f906c1f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 -- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906c15f800 msgr2=0x7f906c09ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906c15f800 0x7f906c09ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.758+0000 7f906a7fc640 1 -- 172.21.15.62:0/4165102620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c1f5e30 con 0x7f906c0d3df0 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f906a7fc640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f906000c4a0 tx=0x7f906000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060018070 con 0x7f906c0d3df0 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90600074e0 con 0x7f906c0d3df0 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c0cd410 con 0x7f906c0d3df0 2024-04-26T22:26:52.761 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060004a20 con 0x7f906c0d3df0 2024-04-26T22:26:52.762 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c0cdcb0 con 0x7f906c0d3df0 2024-04-26T22:26:52.762 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f906001b070 con 0x7f906c0d3df0 2024-04-26T22:26:52.762 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.759+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9060004bc0 con 0x7f906c0d3df0 2024-04-26T22:26:52.762 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.760+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c0cd800 con 0x7f906c0d3df0 2024-04-26T22:26:52.762 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.762+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f906001d440 con 0x7f906c0d3df0 2024-04-26T22:26:52.825 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.825+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f906c0767e0 con 0x7f906c0d3df0 2024-04-26T22:26:52.826 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.825+0000 7f9053fff640 1 -- 172.21.15.62:0/4165102620 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f906001d620 con 0x7f906c0d3df0 2024-04-26T22:26:52.826 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-26T22:26:52.826 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":10,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:26:51.598583+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 msgr2=0x7f906c0a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f906000c4a0 tx=0x7f906000c960 comp rx=0 tx=0).stop 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 shutdown_connections 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f906c15f800 0x7f906c09ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f906c0d3df0 0x7f906c0a11f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.827 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 --2- 172.21.15.62:0/4165102620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f906c0d3a10 0x7f906c1f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:52.828 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 >> 172.21.15.62:0/4165102620 conn(0x7f906c09e520 msgr2=0x7f906c06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:52.828 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 shutdown_connections 2024-04-26T22:26:52.828 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:52.826+0000 7f9071904640 1 -- 172.21.15.62:0/4165102620 wait complete. 2024-04-26T22:26:52.836 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T22:26:52.844 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.844+0000 7f5eb4012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T22:26:52.875 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.874+0000 7f9d3d61e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:26:52.923 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.923+0000 7f9d3d61e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:26:52.988 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:52.988+0000 7f5eb4012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T22:26:52.991 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:52.991+0000 7f9d3d61e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:26:53.035 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:53.035+0000 7f9d3d61e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:26:53.040 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:53.040+0000 7f5eb4012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T22:26:53.114 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:53.114+0000 7f5eb4012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T22:26:53.162 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:53.162+0000 7f5eb4012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T22:26:53.507 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:53.507+0000 7f9d3d61e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:26:53.625 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:53.624+0000 7f9d3d61e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:26:53.671 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:53.671+0000 7f5eb4012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T22:26:53.792 INFO:tasks.ceph.mgr.y.smithi062.stderr:2024-04-26T22:26:53.792+0000 7f5eb4012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T22:26:54.641 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:54.640+0000 7f9cdd5ae640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T22:26:54.641 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:26:54.640+0000 7f9cdd5ae640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T22:26:55.704 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-26T22:26:55.704+0000 7f7323232640 -1 osd.2 0 log_to_monitors true 2024-04-26T22:26:55.730 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-04-26T22:26:55.729+0000 7f6b0fa2d640 -1 osd.7 0 log_to_monitors true 2024-04-26T22:26:55.750 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-26T22:26:55.751+0000 7f31fdaea640 -1 osd.3 0 log_to_monitors true 2024-04-26T22:26:55.756 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-26T22:26:55.756+0000 7f807872c640 -1 osd.1 0 log_to_monitors true 2024-04-26T22:26:55.762 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-04-26T22:26:55.762+0000 7f6da72af640 -1 osd.4 0 log_to_monitors true 2024-04-26T22:26:55.778 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-04-26T22:26:55.778+0000 7f51adc86640 -1 osd.6 0 log_to_monitors true 2024-04-26T22:26:55.783 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-26T22:26:55.783+0000 7f32275a1640 -1 osd.0 0 log_to_monitors true 2024-04-26T22:26:55.795 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-04-26T22:26:55.795+0000 7f5e3f5ba640 -1 osd.5 0 log_to_monitors true 2024-04-26T22:26:59.644 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T22:26:59.738 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-26T22:26:59.738+0000 7f731f1c8640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T22:26:59.752 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-04-26T22:26:59.751+0000 7f6b0b007640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T22:26:59.756 INFO:tasks.ceph.osd.2.smithi062.stderr:2024-04-26T22:26:59.756+0000 7f7316fe8640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.763 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-04-26T22:26:59.763+0000 7f51a9c1c640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T22:26:59.767 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-04-26T22:26:59.767+0000 7f6b02e27640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.776 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-04-26T22:26:59.775+0000 7f5e3b550640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T22:26:59.776 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-26T22:26:59.776+0000 7f31fa294640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T22:26:59.777 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-04-26T22:26:59.776+0000 7f6da3a59640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T22:26:59.777 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-04-26T22:26:59.777+0000 7f51a1a3c640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.781 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-26T22:26:59.781+0000 7f80746c2640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T22:26:59.782 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.782+0000 7ff59a997640 1 Processor -- start 2024-04-26T22:26:59.782 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.782+0000 7ff59a997640 1 -- start start 2024-04-26T22:26:59.784 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.783+0000 7ff59a997640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff594074280 0x7ff594071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.783+0000 7ff59a997640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff5940cf380 0x7ff5940cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff593fff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7ff59406c350 0x7ff59406c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:41716/0 (socket says 172.21.15.62:41716) 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff594074280 0x7ff594071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff594074280 0x7ff594071140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55212/0 (socket says 172.21.15.62:55212) 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff593fff640 1 -- 172.21.15.62:0/3340368818 learned_addr learned my addr 172.21.15.62:0/3340368818 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff598f34640 1 --2- 172.21.15.62:0/3340368818 >> v2:172.21.15.174:3300/0 conn(0x7ff5940cf380 0x7ff5940cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff594071840 con 0x7ff59406c350 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff59406d1d0 con 0x7ff594074280 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff594071b20 con 0x7ff5940cf380 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 -- 172.21.15.62:0/3340368818 >> v2:172.21.15.174:3300/0 conn(0x7ff5940cf380 msgr2=0x7ff5940cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 --2- 172.21.15.62:0/3340368818 >> v2:172.21.15.174:3300/0 conn(0x7ff5940cf380 0x7ff5940cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 -- 172.21.15.62:0/3340368818 >> v1:172.21.15.174:6789/0 conn(0x7ff59406c350 legacy=0x7ff59406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.785 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.784+0000 7ff5937fe640 1 -- 172.21.15.62:0/3340368818 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5940ce8a0 con 0x7ff594074280 2024-04-26T22:26:59.786 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.785+0000 7ff5937fe640 1 --2- 172.21.15.62:0/3340368818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff594074280 0x7ff594071140 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff594068e70 tx=0x7ff58802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e325fc3b1910d530 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:59.786 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.786+0000 7ff592ffd640 1 -- 172.21.15.62:0/3340368818 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588037070 con 0x7ff594074280 2024-04-26T22:26:59.786 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.786+0000 7ff592ffd640 1 -- 172.21.15.62:0/3340368818 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff588032440 con 0x7ff594074280 2024-04-26T22:26:59.786 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.786+0000 7ff592ffd640 1 -- 172.21.15.62:0/3340368818 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588032810 con 0x7ff594074280 2024-04-26T22:26:59.787 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff594074280 msgr2=0x7ff594071140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.787 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 --2- 172.21.15.62:0/3340368818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff594074280 0x7ff594071140 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff594068e70 tx=0x7ff58802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 shutdown_connections 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 --2- 172.21.15.62:0/3340368818 >> v2:172.21.15.174:3300/0 conn(0x7ff5940cf380 0x7ff5940cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 --2- 172.21.15.62:0/3340368818 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff594074280 0x7ff594071140 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 >> 172.21.15.62:0/3340368818 conn(0x7ff5940b3cb0 msgr2=0x7ff5940b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 shutdown_connections 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.787+0000 7ff59a997640 1 -- 172.21.15.62:0/3340368818 wait complete. 2024-04-26T22:26:59.788 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.789+0000 7ff59a997640 1 Processor -- start 2024-04-26T22:26:59.789 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.789+0000 7ff59a997640 1 -- start start 2024-04-26T22:26:59.789 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.790 INFO:tasks.ceph.osd.3.smithi062.stderr:2024-04-26T22:26:59.790+0000 7f31f18a0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.791 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff594074280 0x7ff5940b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.791 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.791 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5940cf380 0x7ff594160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.791 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff594143b70 con 0x7ff59406c350 2024-04-26T22:26:59.791 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940cee60 con 0x7ff594074280 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff59a997640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff594143cf0 con 0x7ff5940cf380 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:55214/0 (socket says 172.21.15.62:55214) 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 -- 172.21.15.62:0/803679479 learned_addr learned my addr 172.21.15.62:0/803679479 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff5937fe640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff594074280 0x7ff5940b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff598f34640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5940cf380 0x7ff594160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 -- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5940cf380 msgr2=0x7ff594160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5940cf380 0x7ff594160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 -- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff594074280 msgr2=0x7ff5940b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff594074280 0x7ff5940b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.790+0000 7ff593fff640 1 -- 172.21.15.62:0/803679479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff594146b80 con 0x7ff59406c350 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff593fff640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c480 tx=0x7ff58400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584017070 con 0x7ff59406c350 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584005ce0 con 0x7ff59406c350 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840072f0 con 0x7ff59406c350 2024-04-26T22:26:59.792 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff594146e60 con 0x7ff59406c350 2024-04-26T22:26:59.793 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.791+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5941442c0 con 0x7ff59406c350 2024-04-26T22:26:59.793 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-04-26T22:26:59.792+0000 7f5e33370640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.793 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.792+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5940767c0 con 0x7ff59406c350 2024-04-26T22:26:59.794 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-04-26T22:26:59.793+0000 7f6d9b065640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.794 INFO:tasks.ceph.osd.1.smithi062.stderr:2024-04-26T22:26:59.793+0000 7f806c4e2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.795 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.793+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff58401a070 con 0x7ff59406c350 2024-04-26T22:26:59.795 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.793+0000 7ff590ff9640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 0x7ff58006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:26:59.795 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.794+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7ff584089fb0 con 0x7ff59406c350 2024-04-26T22:26:59.795 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.794+0000 7ff5937fe640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 0x7ff58006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:26:59.795 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.794+0000 7ff5937fe640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 0x7ff58006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:26:59.799 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.800+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff584058bf0 con 0x7ff59406c350 2024-04-26T22:26:59.812 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-26T22:26:59.812+0000 7f3223537640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T22:26:59.824 INFO:tasks.ceph.osd.0.smithi062.stderr:2024-04-26T22:26:59.824+0000 7f321b357640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T22:26:59.990 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.990+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff594146890 con 0x7ff59406c350 2024-04-26T22:26:59.990 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.990+0000 7ff590ff9640 1 -- 172.21.15.62:0/803679479 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7ff58405cef0 con 0x7ff59406c350 2024-04-26T22:26:59.990 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-26T22:26:59.991 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":12,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:26:57.640591+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 msgr2=0x7ff58006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 0x7ff58006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff588034000 tx=0x7ff58802e000 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 msgr2=0x7ff594146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff58400c480 tx=0x7ff58400c940 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 shutdown_connections 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff58006c860 0x7ff58006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5940cf380 0x7ff594160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff594074280 0x7ff5940b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 --2- 172.21.15.62:0/803679479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff59406c350 0x7ff594146350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 >> 172.21.15.62:0/803679479 conn(0x7ff5940b3cb0 msgr2=0x7ff5940cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 shutdown_connections 2024-04-26T22:26:59.993 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:26:59.992+0000 7ff59a997640 1 -- 172.21.15.62:0/803679479 wait complete. 2024-04-26T22:27:00.002 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T22:27:06.809 DEBUG:teuthology.orchestra.run.smithi062:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:06.914 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.914+0000 7fd6bedc6640 1 Processor -- start 2024-04-26T22:27:06.914 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.914+0000 7fd6bedc6640 1 -- start start 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.914+0000 7fd6bedc6640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 0x7fd6b80ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.914+0000 7fd6bedc6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b80b5620 0x7fd6b809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6bedc6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b80cebd0 con 0x7fd6b809bf80 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6bedc6640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b8070a50 con 0x7fd6b80b5620 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6bedc6640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b8070400 con 0x7fd6b815d4c0 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b80b5620 0x7fd6b809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd6b80b5620 0x7fd6b809ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50786/0 (socket says 172.21.15.62:50786) 2024-04-26T22:27:06.917 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 -- 172.21.15.62:0/3420210180 learned_addr learned my addr 172.21.15.62:0/3420210180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6bcb62640 1 --2- 172.21.15.62:0/3420210180 >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 0x7fd6b80ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 -- 172.21.15.62:0/3420210180 >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 msgr2=0x7fd6b80ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 --2- 172.21.15.62:0/3420210180 >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 0x7fd6b80ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 -- 172.21.15.62:0/3420210180 >> v1:172.21.15.174:6789/0 conn(0x7fd6b809bf80 legacy=0x7fd6b80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6affff640 1 -- 172.21.15.62:0/3420210180 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b80706e0 con 0x7fd6b80b5620 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.915+0000 7fd6bcb62640 1 --2- 172.21.15.62:0/3420210180 >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 0x7fd6b80ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6affff640 1 --2- 172.21.15.62:0/3420210180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80b5620 0x7fd6b809ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6b806cdf0 tx=0x7fd6a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5cddf30e6c3105 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6af7fe640 1 -- 172.21.15.62:0/3420210180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8037070 con 0x7fd6b80b5620 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6af7fe640 1 -- 172.21.15.62:0/3420210180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6a8032440 con 0x7fd6b80b5620 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6af7fe640 1 -- 172.21.15.62:0/3420210180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8032810 con 0x7fd6b80b5620 2024-04-26T22:27:06.918 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3420210180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80b5620 msgr2=0x7fd6b809ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3420210180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80b5620 0x7fd6b809ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd6b806cdf0 tx=0x7fd6a802a230 comp rx=0 tx=0).stop 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3420210180 shutdown_connections 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.916+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3420210180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b80b5620 0x7fd6b809ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.917+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3420210180 >> v2:172.21.15.174:3300/0 conn(0x7fd6b815d4c0 0x7fd6b80ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.917+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3420210180 >> 172.21.15.62:0/3420210180 conn(0x7fd6b80b4b00 msgr2=0x7fd6b80b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.917+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3420210180 shutdown_connections 2024-04-26T22:27:06.919 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.917+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3420210180 wait complete. 2024-04-26T22:27:06.922 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.918+0000 7fd6bedc6640 1 Processor -- start 2024-04-26T22:27:06.922 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.918+0000 7fd6bedc6640 1 -- start start 2024-04-26T22:27:06.922 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6b809bf80 0x7fd6b81577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.922 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6b80b5620 0x7fd6b814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80d2010 con 0x7fd6b815d4c0 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80d1e90 con 0x7fd6b809bf80 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bedc6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8071ef0 con 0x7fd6b80b5620 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50790/0 (socket says 172.21.15.62:50790) 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 -- 172.21.15.62:0/3746060468 learned_addr learned my addr 172.21.15.62:0/3746060468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bcb62640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6b809bf80 0x7fd6b81577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6affff640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6b80b5620 0x7fd6b814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 -- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6b80b5620 msgr2=0x7fd6b814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6b80b5620 0x7fd6b814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 -- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6b809bf80 msgr2=0x7fd6b81577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6b809bf80 0x7fd6b81577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 -- 172.21.15.62:0/3746060468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b815dde0 con 0x7fd6b815d4c0 2024-04-26T22:27:06.923 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.919+0000 7fd6bd363640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c7b0 tx=0x7fd6b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.920+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0004b60 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.920+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0004d00 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.920+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b80b7540 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.920+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000aa90 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.920+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b80b6f80 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.921+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b806d230 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.921+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6b000ac30 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.922+0000 7fd6ad7fa640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 0x7fd6a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.922+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd6b008bc80 con 0x7fd6b815d4c0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.922+0000 7fd6bcb62640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 0x7fd6a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:06.924 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.922+0000 7fd6bcb62640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 0x7fd6a406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8158080 tx=0x7fd6b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:06.926 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:06.926+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd6b005a490 con 0x7fd6b815d4c0 2024-04-26T22:27:07.113 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.113+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6b80c92b0 con 0x7fd6b815d4c0 2024-04-26T22:27:07.114 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.114+0000 7fd6ad7fa640 1 -- 172.21.15.62:0/3746060468 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fd6b0059d90 con 0x7fd6b815d4c0 2024-04-26T22:27:07.115 INFO:teuthology.misc.health.smithi062.stdout: 2024-04-26T22:27:07.115 INFO:teuthology.misc.health.smithi062.stdout:{"epoch":16,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:03.818329+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.116+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 msgr2=0x7fd6a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.116+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 0x7fd6a406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8158080 tx=0x7fd6b4008040 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.116+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 msgr2=0x7fd6b815d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.116+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c7b0 tx=0x7fd6b000cc70 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 shutdown_connections 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd6a406c920 0x7fd6a406ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6b815d4c0 0x7fd6b815d8a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6b80b5620 0x7fd6b814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 --2- 172.21.15.62:0/3746060468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6b809bf80 0x7fd6b81577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 >> 172.21.15.62:0/3746060468 conn(0x7fd6b80b4b00 msgr2=0x7fd6b80b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.117 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 shutdown_connections 2024-04-26T22:27:07.118 INFO:teuthology.misc.health.smithi062.stderr:2024-04-26T22:27:07.117+0000 7fd6bedc6640 1 -- 172.21.15.62:0/3746060468 wait complete. 2024-04-26T22:27:07.127 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T22:27:07.127 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T22:27:07.127 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:27:07.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:27:07.129 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:27:07.130 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:27:07.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:27:07.132 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:27:07.132 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:27:07.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:27:07.135 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:27:07.135 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T22:27:07.135 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T22:27:07.137 INFO:tasks.ceph.mds.d:Started 2024-04-26T22:27:07.137 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T22:27:07.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:07.205 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-26T22:27:07.205 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-26T22:27:07.218 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2024-04-26T22:27:07.219 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2024-04-26T22:27:07.219 INFO:tasks.ceph.mds.b.smithi174.stderr:2024-04-26T22:27:07.219+0000 7f96de4e3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T22:27:07.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.242+0000 7fa300e99640 1 Processor -- start 2024-04-26T22:27:07.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.242+0000 7fa300e99640 1 -- start start 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa300e99640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2fc073800 0x7fa2fc0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa300e99640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa2fc0712f0 0x7fa2fc0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa300e99640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2fc070250 con 0x7fa2fc144c50 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa300e99640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc070530 con 0x7fa2fc073800 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa300e99640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa2fc070810 con 0x7fa2fc0712f0 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa2f9d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2fc073800 0x7fa2fc0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa2fa59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa2fc144c50 0x7fa2fc0730f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57684/0 (socket says 172.21.15.62:57684) 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.243+0000 7fa2fa59c640 1 -- 172.21.15.62:0/1046693794 learned_addr learned my addr 172.21.15.62:0/1046693794 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2fad9d640 1 --2- 172.21.15.62:0/1046693794 >> v2:172.21.15.174:3300/0 conn(0x7fa2fc0712f0 0x7fa2fc0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/1046693794 >> v2:172.21.15.174:3300/0 conn(0x7fa2fc0712f0 msgr2=0x7fa2fc0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f959a640 1 -- 172.21.15.62:0/1046693794 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465011783 0 0) 0x7fa2fc070250 con 0x7fa2fc144c50 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f9d9b640 1 --2- 172.21.15.62:0/1046693794 >> v2:172.21.15.174:3300/0 conn(0x7fa2fc0712f0 0x7fa2fc0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/1046693794 >> v1:172.21.15.62:6789/0 conn(0x7fa2fc144c50 legacy=0x7fa2fc0730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/1046693794 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc070af0 con 0x7fa2fc073800 2024-04-26T22:27:07.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.244+0000 7fa2f9d9b640 1 --2- 172.21.15.62:0/1046693794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc0b6770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0ceb40 tx=0x7fa2f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78be2fb5b965b0b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa2f959a640 1 -- 172.21.15.62:0/1046693794 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0037070 con 0x7fa2fc073800 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa2f959a640 1 -- 172.21.15.62:0/1046693794 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f0032440 con 0x7fa2fc073800 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa2f959a640 1 -- 172.21.15.62:0/1046693794 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f002ae60 con 0x7fa2fc073800 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 -- 172.21.15.62:0/1046693794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 msgr2=0x7fa2fc0b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 --2- 172.21.15.62:0/1046693794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc0b6770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc0ceb40 tx=0x7fa2f002a110 comp rx=0 tx=0).stop 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 -- 172.21.15.62:0/1046693794 shutdown_connections 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 --2- 172.21.15.62:0/1046693794 >> v2:172.21.15.174:3300/0 conn(0x7fa2fc0712f0 0x7fa2fc0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 --2- 172.21.15.62:0/1046693794 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc0b6770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 -- 172.21.15.62:0/1046693794 >> 172.21.15.62:0/1046693794 conn(0x7fa2fc0b3b00 msgr2=0x7fa2fc0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 -- 172.21.15.62:0/1046693794 shutdown_connections 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.245+0000 7fa300e99640 1 -- 172.21.15.62:0/1046693794 wait complete. 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 Processor -- start 2024-04-26T22:27:07.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 -- start start 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 0x7fa2fc1f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2fc144c50 0x7fa2fc0d0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc145030 con 0x7fa2fc073800 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc145b00 con 0x7fa2fc0712f0 2024-04-26T22:27:07.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa300e99640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa2fc1451b0 con 0x7fa2fc144c50 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa2f9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50840/0 (socket says 172.21.15.62:50840) 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.247+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/4029618264 learned_addr learned my addr 172.21.15.62:0/4029618264 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2fad9d640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2fc144c50 0x7fa2fc0d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2fa59c640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 0x7fa2fc1f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2fc144c50 msgr2=0x7fa2fc0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2fc144c50 0x7fa2fc0d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 msgr2=0x7fa2fc1f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 0x7fa2fc1f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 -- 172.21.15.62:0/4029618264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc15e860 con 0x7fa2fc073800 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2fa59c640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 0x7fa2fc1f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2f9d9b640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000c00 tx=0x7fa2f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0037050 con 0x7fa2fc073800 2024-04-26T22:27:07.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f0038040 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc13b170 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.248+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc15e580 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.249+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0004450 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.249+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc13b340 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.250+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2f003c6d0 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.250+0000 7fa2eaffd640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 0x7fa2dc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.250+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa2f0004230 con 0x7fa2fc073800 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.251+0000 7fa2fa59c640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 0x7fa2dc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.251+0000 7fa2fa59c640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 0x7fa2dc06efb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000f60 tx=0x7fa2ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.255+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2f007a7a0 con 0x7fa2fc073800 2024-04-26T22:27:07.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.441+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2fc0693e0 con 0x7fa2fc073800 2024-04-26T22:27:07.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.442+0000 7fa2eaffd640 1 -- 172.21.15.62:0/4029618264 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fa2f007a0a0 con 0x7fa2fc073800 2024-04-26T22:27:07.442 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:07.442 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:03.818329+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:07.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 msgr2=0x7fa2dc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 0x7fa2dc06efb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000f60 tx=0x7fa2ec009040 comp rx=0 tx=0).stop 2024-04-26T22:27:07.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 msgr2=0x7fa2fc15d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0000c00 tx=0x7fa2f00046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 shutdown_connections 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06caf0 0x7fa2dc06efb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2fc144c50 0x7fa2fc0d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2fc073800 0x7fa2fc15d6c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 --2- 172.21.15.62:0/4029618264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc0712f0 0x7fa2fc1f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 >> 172.21.15.62:0/4029618264 conn(0x7fa2fc0b3b00 msgr2=0x7fa2fc141680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 shutdown_connections 2024-04-26T22:27:07.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.444+0000 7fa300e99640 1 -- 172.21.15.62:0/4029618264 wait complete. 2024-04-26T22:27:07.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:07.455 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.558+0000 7f38ae2d9640 1 Processor -- start 2024-04-26T22:27:07.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.558+0000 7f38ae2d9640 1 -- start start 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ae2d9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f38a8160800 0x7f38a81610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ae2d9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a80d2320 0x7f38a80d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ae2d9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a80d0930 con 0x7f38a80d5b60 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ae2d9640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80d0c00 con 0x7f38a8160800 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ae2d9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f38a80d0ee0 con 0x7f38a80d2320 2024-04-26T22:27:07.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a7fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f38a8160800 0x7f38a81610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a77fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f38a80d5b60 0x7f38a80d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57690/0 (socket says 172.21.15.62:57690) 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a77fe640 1 -- 172.21.15.62:0/3596168520 learned_addr learned my addr 172.21.15.62:0/3596168520 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38ac876640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a80d2320 0x7f38a80d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a7fff640 1 -- 172.21.15.62:0/3596168520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a80d2320 msgr2=0x7f38a80d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a7fff640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a80d2320 0x7f38a80d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.559+0000 7f38a7fff640 1 -- 172.21.15.62:0/3596168520 >> v1:172.21.15.62:6789/0 conn(0x7f38a80d5b60 legacy=0x7f38a80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.560+0000 7f38a7fff640 1 -- 172.21.15.62:0/3596168520 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80d11c0 con 0x7f38a8160800 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.560+0000 7f38a7fff640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a8160800 0x7f38a81610c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38900097a0 tx=0x7f389002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7599cb9e31a8b831 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.560+0000 7f38a6ffd640 1 -- 172.21.15.62:0/3596168520 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890037070 con 0x7f38a8160800 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.560+0000 7f38a6ffd640 1 -- 172.21.15.62:0/3596168520 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389002ad10 con 0x7f38a8160800 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.560+0000 7f38a6ffd640 1 -- 172.21.15.62:0/3596168520 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890032660 con 0x7f38a8160800 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3596168520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a8160800 msgr2=0x7f38a81610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.562 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a8160800 0x7f38a81610c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f38900097a0 tx=0x7f389002a080 comp rx=0 tx=0).stop 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3596168520 shutdown_connections 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a80d2320 0x7f38a80d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3596168520 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a8160800 0x7f38a81610c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3596168520 >> 172.21.15.62:0/3596168520 conn(0x7f38a809e550 msgr2=0x7f38a80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3596168520 shutdown_connections 2024-04-26T22:27:07.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.561+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3596168520 wait complete. 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.562+0000 7f38ae2d9640 1 Processor -- start 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.562+0000 7f38ae2d9640 1 -- start start 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a80d5b60 0x7f38a81f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 0x7f38a80ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80b6300 con 0x7f38a80d2320 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80d30f0 con 0x7f38a80d5b60 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ae2d9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f38a8160be0 con 0x7f38a8160800 2024-04-26T22:27:07.566 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50864/0 (socket says 172.21.15.62:50864) 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 -- 172.21.15.62:0/3626249181 learned_addr learned my addr 172.21.15.62:0/3626249181 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ac876640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 0x7f38a80ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a77fe640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a80d5b60 0x7f38a81f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 -- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 msgr2=0x7f38a80ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 0x7f38a80ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 -- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a80d5b60 msgr2=0x7f38a81f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a80d5b60 0x7f38a81f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 -- 172.21.15.62:0/3626249181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80cc8c0 con 0x7f38a80d2320 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38ac876640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 0x7f38a80ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.563+0000 7f38a7fff640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3890000c00 tx=0x7f38900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.564+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890037040 con 0x7f38a80d2320 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.564+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3890004250 con 0x7f38a80d2320 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.564+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a80ccba0 con 0x7f38a80d2320 2024-04-26T22:27:07.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.564+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890003610 con 0x7f38a80d2320 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.564+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80cd160 con 0x7f38a80d2320 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.565+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a806a360 con 0x7f38a80d2320 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.565+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f389002abd0 con 0x7f38a80d2320 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.566+0000 7f38a4ff9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 0x7f388806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.566+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f38900ac220 con 0x7f38a80d2320 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.566+0000 7f38a77fe640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 0x7f388806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.566+0000 7f38a77fe640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 0x7f388806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3898007fd0 tx=0x7f3898009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.570+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f389007a9a0 con 0x7f38a80d2320 2024-04-26T22:27:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.753+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38a80d4530 con 0x7f38a80d2320 2024-04-26T22:27:07.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.753+0000 7f38a4ff9640 1 -- 172.21.15.62:0/3626249181 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f389007a2a0 con 0x7f38a80d2320 2024-04-26T22:27:07.754 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 msgr2=0x7f388806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 0x7f388806edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3898007fd0 tx=0x7f3898009040 comp rx=0 tx=0).stop 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 msgr2=0x7f38a81f5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3890000c00 tx=0x7f38900046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 shutdown_connections 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f388806c8f0 0x7f388806edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38a8160800 0x7f38a80ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38a80d5b60 0x7f38a81f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 --2- 172.21.15.62:0/3626249181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38a80d2320 0x7f38a81f5540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.755+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 >> 172.21.15.62:0/3626249181 conn(0x7f38a809e550 msgr2=0x7f38a81620b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.756+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 shutdown_connections 2024-04-26T22:27:07.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.756+0000 7f38ae2d9640 1 -- 172.21.15.62:0/3626249181 wait complete. 2024-04-26T22:27:07.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:07.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.867+0000 7f8455951640 1 Processor -- start 2024-04-26T22:27:07.867 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.867+0000 7f8455951640 1 -- start start 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f8455951640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84500d3820 0x7f84500d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f8455951640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500b4ba0 0x7f84500b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f8455951640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500d0210 con 0x7f8450162b40 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f8455951640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f84500d41c0 con 0x7f84500d3820 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f8455951640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f84500d04f0 con 0x7f84500b4ba0 2024-04-26T22:27:07.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84500d3820 0x7f84500d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84500d3820 0x7f84500d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50866/0 (socket says 172.21.15.62:50866) 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 -- 172.21.15.62:0/3150270679 learned_addr learned my addr 172.21.15.62:0/3150270679 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844f7fe640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500b4ba0 0x7f84500b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 -- 172.21.15.62:0/3150270679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500b4ba0 msgr2=0x7f84500b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500b4ba0 0x7f84500b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 -- 172.21.15.62:0/3150270679 >> v1:172.21.15.174:6789/0 conn(0x7f8450162b40 legacy=0x7f84500d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.868+0000 7f844e7fc640 1 -- 172.21.15.62:0/3150270679 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500d07d0 con 0x7f84500d3820 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.869+0000 7f844e7fc640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500d3820 0x7f84500d3c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8444009870 tx=0x7f844402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2eb01a3f9c82a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.869+0000 7f844dffb640 1 -- 172.21.15.62:0/3150270679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444037070 con 0x7f84500d3820 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.869+0000 7f844dffb640 1 -- 172.21.15.62:0/3150270679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8444032440 con 0x7f84500d3820 2024-04-26T22:27:07.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.869+0000 7f844dffb640 1 -- 172.21.15.62:0/3150270679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444032810 con 0x7f84500d3820 2024-04-26T22:27:07.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 -- 172.21.15.62:0/3150270679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500d3820 msgr2=0x7f84500d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500d3820 0x7f84500d3c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8444009870 tx=0x7f844402a270 comp rx=0 tx=0).stop 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 -- 172.21.15.62:0/3150270679 shutdown_connections 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500b4ba0 0x7f84500b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 --2- 172.21.15.62:0/3150270679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500d3820 0x7f84500d3c80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 -- 172.21.15.62:0/3150270679 >> 172.21.15.62:0/3150270679 conn(0x7f845009e550 msgr2=0x7f84500d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 -- 172.21.15.62:0/3150270679 shutdown_connections 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.870+0000 7f8455951640 1 -- 172.21.15.62:0/3150270679 wait complete. 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.871+0000 7f8455951640 1 Processor -- start 2024-04-26T22:27:07.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.871+0000 7f8455951640 1 -- start start 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500d3820 0x7f84500b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8450162b40 0x7f84500b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84500d12b0 con 0x7f84500b4ba0 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84500d0d90 con 0x7f8450162b40 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f8455951640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f84500cfe30 con 0x7f84500d3820 2024-04-26T22:27:07.874 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50870/0 (socket says 172.21.15.62:50870) 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 -- 172.21.15.62:0/3198141387 learned_addr learned my addr 172.21.15.62:0/3198141387 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844e7fc640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500d3820 0x7f84500b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844f7fe640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8450162b40 0x7f84500b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 -- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500d3820 msgr2=0x7f84500b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500d3820 0x7f84500b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 -- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8450162b40 msgr2=0x7f84500b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8450162b40 0x7f84500b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.872+0000 7f844effd640 1 -- 172.21.15.62:0/3198141387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500cc5f0 con 0x7f84500b4ba0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f844effd640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f844800c400 tx=0x7f844800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480052d0 con 0x7f84500b4ba0 2024-04-26T22:27:07.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8448005490 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8448007260 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84500b9470 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.873+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84500b9a30 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.874+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845006a360 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.874+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8448007400 con 0x7f84500b4ba0 2024-04-26T22:27:07.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.874+0000 7f8437fff640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.875+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f844808acd0 con 0x7f84500b4ba0 2024-04-26T22:27:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.875+0000 7f844e7fc640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:07.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.875+0000 7f844e7fc640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8444034000 tx=0x7f844402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:07.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:07.879+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8448058e10 con 0x7f84500b4ba0 2024-04-26T22:27:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.110+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84500d2960 con 0x7f84500b4ba0 2024-04-26T22:27:08.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.110+0000 7f8437fff640 1 -- 172.21.15.62:0/3198141387 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f844805d110 con 0x7f84500b4ba0 2024-04-26T22:27:08.110 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:08.111 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T22:27:08.111 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 2 2024-04-26T22:27:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 msgr2=0x7f843c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 0x7f843c06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8444034000 tx=0x7f844402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 msgr2=0x7f84500baa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:08.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f844800c400 tx=0x7f844800c8c0 comp rx=0 tx=0).stop 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 shutdown_connections 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f843c06c8f0 0x7f843c06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8450162b40 0x7f84500b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84500d3820 0x7f84500b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 --2- 172.21.15.62:0/3198141387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84500b4ba0 0x7f84500baa70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 >> 172.21.15.62:0/3198141387 conn(0x7f845009e550 msgr2=0x7f845006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 shutdown_connections 2024-04-26T22:27:08.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:08.112+0000 7f8455951640 1 -- 172.21.15.62:0/3198141387 wait complete. 2024-04-26T22:27:10.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:10.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.237+0000 7f7c91151640 1 Processor -- start 2024-04-26T22:27:10.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.237+0000 7f7c91151640 1 -- start start 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c91151640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7c8c15f380 0x7f7c8c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c91151640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8c0d3de0 con 0x7f7c8c15f760 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c91151640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c8c0d40c0 con 0x7f7c8c0b9a90 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c91151640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f7c8c0d43a0 con 0x7f7c8c15f380 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c8a59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7c8c15f760 0x7f7c8c0cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57702/0 (socket says 172.21.15.62:57702) 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c8a59c640 1 -- 172.21.15.62:0/3684196736 learned_addr learned my addr 172.21.15.62:0/3684196736 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:10.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/3684196736 >> v2:172.21.15.174:3300/0 conn(0x7f7c8c15f380 0x7f7c8c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.238+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855434723 0 0) 0x7f7c8c0d3de0 con 0x7f7c8c15f760 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c78003520 con 0x7f7c8c15f760 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2597755469 0 0) 0x7f7c8c0d40c0 con 0x7f7c8c0b9a90 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c8c0d3de0 con 0x7f7c8c0b9a90 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 613842276 0 0) 0x7f7c78003520 con 0x7f7c8c15f760 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c8ad9d640 1 -- 172.21.15.62:0/3684196736 >> v1:172.21.15.174:6789/0 conn(0x7f7c8c0b9a90 legacy=0x7f7c8c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c8ad9d640 1 -- 172.21.15.62:0/3684196736 >> v1:172.21.15.62:6789/0 conn(0x7f7c8c15f760 legacy=0x7f7c8c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.239+0000 7f7c8ad9d640 1 -- 172.21.15.62:0/3684196736 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c0b8280 con 0x7f7c8c15f380 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/3684196736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0d1bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097a0 tx=0x7f7c8402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431df2ebba0184 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84037070 con 0x7f7c8c15f380 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8402ace0 con 0x7f7c8c15f380 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c89d9b640 1 -- 172.21.15.62:0/3684196736 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74001670 con 0x7f7c8c15f380 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 -- 172.21.15.62:0/3684196736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c15f380 msgr2=0x7f7c8c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 --2- 172.21.15.62:0/3684196736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0d1bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c840097a0 tx=0x7f7c8402a250 comp rx=0 tx=0).stop 2024-04-26T22:27:10.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 -- 172.21.15.62:0/3684196736 shutdown_connections 2024-04-26T22:27:10.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 --2- 172.21.15.62:0/3684196736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0d1bd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 -- 172.21.15.62:0/3684196736 >> 172.21.15.62:0/3684196736 conn(0x7f7c8c09e560 msgr2=0x7f7c8c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:10.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.240+0000 7f7c91151640 1 -- 172.21.15.62:0/3684196736 shutdown_connections 2024-04-26T22:27:10.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.241+0000 7f7c91151640 1 -- 172.21.15.62:0/3684196736 wait complete. 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.242+0000 7f7c91151640 1 Processor -- start 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.242+0000 7f7c91151640 1 -- start start 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 0x7f7c8c0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 0x7f7c8c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0d1100 con 0x7f7c8c15f380 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0d0cc0 con 0x7f7c8c0b9a90 2024-04-26T22:27:10.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c91151640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8c0d1280 con 0x7f7c8c15f760 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50880/0 (socket says 172.21.15.62:50880) 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 -- 172.21.15.62:0/2580282868 learned_addr learned my addr 172.21.15.62:0/2580282868 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 0x7f7c8c0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8b59e640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 0x7f7c8c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 -- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 msgr2=0x7f7c8c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 0x7f7c8c15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 -- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 msgr2=0x7f7c8c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 0x7f7c8c0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8a59c640 1 -- 172.21.15.62:0/2580282868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c15cd00 con 0x7f7c8c15f380 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8b59e640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 0x7f7c8c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.243+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 0x7f7c8c0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c8a59c640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c74000fd0 tx=0x7f7c7400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74017070 con 0x7f7c8c15f380 2024-04-26T22:27:10.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c74005ad0 con 0x7f7c8c15f380 2024-04-26T22:27:10.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c0cc900 con 0x7f7c8c15f380 2024-04-26T22:27:10.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74007070 con 0x7f7c8c15f380 2024-04-26T22:27:10.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.244+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c0ccec0 con 0x7f7c8c15f380 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.245+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c06a360 con 0x7f7c8c15f380 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.245+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c7401a070 con 0x7f7c8c15f380 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.246+0000 7f7c837fe640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 0x7f7c6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.246+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7c7408abb0 con 0x7f7c8c15f380 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.246+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 0x7f7c6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.247+0000 7f7c8ad9d640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 0x7f7c6c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c84034000 tx=0x7f7c8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.250+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c74058c50 con 0x7f7c8c15f380 2024-04-26T22:27:10.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.473+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c8c0d2f30 con 0x7f7c8c15f380 2024-04-26T22:27:10.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.473+0000 7f7c837fe640 1 -- 172.21.15.62:0/2580282868 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f7c7405cf50 con 0x7f7c8c15f380 2024-04-26T22:27:10.473 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:10.474 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T22:27:10.474 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 3 2024-04-26T22:27:10.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 msgr2=0x7f7c6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 0x7f7c6c06edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c84034000 tx=0x7f7c8402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 msgr2=0x7f7c8c0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c74000fd0 tx=0x7f7c7400cbe0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 shutdown_connections 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7c6c06c8f0 0x7f7c6c06edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c8c15f760 0x7f7c8c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7c8c15f380 0x7f7c8c0cd9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 --2- 172.21.15.62:0/2580282868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8c0b9a90 0x7f7c8c0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 >> 172.21.15.62:0/2580282868 conn(0x7f7c8c09e560 msgr2=0x7f7c8c161c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 shutdown_connections 2024-04-26T22:27:10.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.475+0000 7f7c91151640 1 -- 172.21.15.62:0/2580282868 wait complete. 2024-04-26T22:27:10.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:10.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.586+0000 7f08f5524640 1 Processor -- start 2024-04-26T22:27:10.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.586+0000 7f08f5524640 1 -- start start 2024-04-26T22:27:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 0x7f08f00d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08f015f760 0x7f08f00d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f08f00d43a0 con 0x7f08f015f760 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f08f00d3de0 con 0x7f08f015f380 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08f5524640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f08f00d40c0 con 0x7f08f00d5b50 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08ee7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08f015f760 0x7f08f00d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08ee7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f08f015f760 0x7f08f00d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50884/0 (socket says 172.21.15.62:50884) 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08ee7fc640 1 -- 172.21.15.62:0/536657338 learned_addr learned my addr 172.21.15.62:0/536657338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08eeffd640 1 --2- 172.21.15.62:0/536657338 >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 0x7f08f00d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.587+0000 7f08ef7fe640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 -- 172.21.15.62:0/536657338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 msgr2=0x7f08f00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f00cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 -- 172.21.15.62:0/536657338 >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 msgr2=0x7f08f00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/536657338 >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 0x7f08f00d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 -- 172.21.15.62:0/536657338 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f00b8280 con 0x7f08f015f760 2024-04-26T22:27:10.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08eeffd640 1 --2- 172.21.15.62:0/536657338 >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 0x7f08f00d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f760 0x7f08f00d2110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08f008eb70 tx=0x7f08d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4f6dc3de24dca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08edffb640 1 -- 172.21.15.62:0/536657338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8037070 con 0x7f08f015f760 2024-04-26T22:27:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08edffb640 1 -- 172.21.15.62:0/536657338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d8032440 con 0x7f08f015f760 2024-04-26T22:27:10.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.588+0000 7f08edffb640 1 -- 172.21.15.62:0/536657338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8032810 con 0x7f08f015f760 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 -- 172.21.15.62:0/536657338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f760 msgr2=0x7f08f00d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f760 0x7f08f00d2110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08f008eb70 tx=0x7f08d802a270 comp rx=0 tx=0).stop 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 -- 172.21.15.62:0/536657338 shutdown_connections 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f00cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 --2- 172.21.15.62:0/536657338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f760 0x7f08f00d2110 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 --2- 172.21.15.62:0/536657338 >> v2:172.21.15.174:3300/0 conn(0x7f08f015f380 0x7f08f00d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 -- 172.21.15.62:0/536657338 >> 172.21.15.62:0/536657338 conn(0x7f08f009e560 msgr2=0x7f08f00d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 -- 172.21.15.62:0/536657338 shutdown_connections 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.589+0000 7f08f5524640 1 -- 172.21.15.62:0/536657338 wait complete. 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.590+0000 7f08f5524640 1 Processor -- start 2024-04-26T22:27:10.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.590+0000 7f08f5524640 1 -- start start 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f01f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 0x7f08f015fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00d1280 con 0x7f08f015f380 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00d1100 con 0x7f08f015f760 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08f5524640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f08f00d3840 con 0x7f08f00d5b50 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50896/0 (socket says 172.21.15.62:50896) 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 -- 172.21.15.62:0/2597047199 learned_addr learned my addr 172.21.15.62:0/2597047199 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ef7fe640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 0x7f08f015fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08eeffd640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f01f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 -- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 msgr2=0x7f08f01f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f01f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 -- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 msgr2=0x7f08f015fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.591+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 0x7f08f015fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08ee7fc640 1 -- 172.21.15.62:0/2597047199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f0161960 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08eeffd640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f01f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08ef7fe640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 0x7f08f015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08ee7fc640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08d802a2b0 tx=0x7f08d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8041020 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d80306c0 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f0161c30 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d8037040 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.592+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01621f0 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.593+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f006a360 con 0x7f08f015f380 2024-04-26T22:27:10.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.593+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08d8044020 con 0x7f08f015f380 2024-04-26T22:27:10.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.594+0000 7f08e77fe640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.594+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f08d80abcd0 con 0x7f08f015f380 2024-04-26T22:27:10.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.594+0000 7f08eeffd640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.594+0000 7f08eeffd640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 0x7f08d006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08f00b8a40 tx=0x7f08e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.598+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f08d804b070 con 0x7f08f015f380 2024-04-26T22:27:10.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.783+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08f00baea0 con 0x7f08f015f380 2024-04-26T22:27:10.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.784+0000 7f08e77fe640 1 -- 172.21.15.62:0/2597047199 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f08d807a4e0 con 0x7f08f015f380 2024-04-26T22:27:10.784 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:10.784 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:03.818329+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 msgr2=0x7f08d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 0x7f08d006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08f00b8a40 tx=0x7f08e8008040 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 msgr2=0x7f08f0161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f08d802a2b0 tx=0x7f08d80046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 shutdown_connections 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08f015f760 0x7f08f015fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08f015f380 0x7f08f0161420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 --2- 172.21.15.62:0/2597047199 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f08f00d5b50 0x7f08f01f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 >> 172.21.15.62:0/2597047199 conn(0x7f08f009e560 msgr2=0x7f08f00b9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:10.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 shutdown_connections 2024-04-26T22:27:10.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.786+0000 7f08f5524640 1 -- 172.21.15.62:0/2597047199 wait complete. 2024-04-26T22:27:10.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:10.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:10.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.901+0000 7fb53d79d640 1 Processor -- start 2024-04-26T22:27:10.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.901+0000 7fb53d79d640 1 -- start start 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb53d79d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb538160800 0x7fb5381610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb53d79d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5380d0930 con 0x7fb5380d5b60 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb53d79d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5380d0c00 con 0x7fb5380d2320 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb53d79d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fb5380d0ee0 con 0x7fb538160800 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb536ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb538160800 0x7fb5381610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb5367fc640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb5380d5b60 0x7fb5380d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57712/0 (socket says 172.21.15.62:57712) 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb5367fc640 1 -- 172.21.15.62:0/4124009788 learned_addr learned my addr 172.21.15.62:0/4124009788 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb536ffd640 1 -- 172.21.15.62:0/4124009788 >> v1:172.21.15.174:6789/0 conn(0x7fb5380d2320 legacy=0x7fb5380d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb536ffd640 1 -- 172.21.15.62:0/4124009788 >> v1:172.21.15.62:6789/0 conn(0x7fb5380d5b60 legacy=0x7fb5380d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb536ffd640 1 -- 172.21.15.62:0/4124009788 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380d11c0 con 0x7fb538160800 2024-04-26T22:27:10.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.902+0000 7fb535ffb640 1 -- 172.21.15.62:0/4124009788 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662133709 0 0) 0x7fb5380d0930 con 0x7fb5380d5b60 2024-04-26T22:27:10.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.903+0000 7fb536ffd640 1 --2- 172.21.15.62:0/4124009788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb538160800 0x7fb5381610c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097a0 tx=0x7fb53002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fb9fb775786c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.903+0000 7fb535ffb640 1 -- 172.21.15.62:0/4124009788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530037070 con 0x7fb538160800 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.903+0000 7fb535ffb640 1 -- 172.21.15.62:0/4124009788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53002ae50 con 0x7fb538160800 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.903+0000 7fb535ffb640 1 -- 172.21.15.62:0/4124009788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530032660 con 0x7fb538160800 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 -- 172.21.15.62:0/4124009788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb538160800 msgr2=0x7fb5381610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 --2- 172.21.15.62:0/4124009788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb538160800 0x7fb5381610c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097a0 tx=0x7fb53002a140 comp rx=0 tx=0).stop 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 -- 172.21.15.62:0/4124009788 shutdown_connections 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 --2- 172.21.15.62:0/4124009788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb538160800 0x7fb5381610c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 -- 172.21.15.62:0/4124009788 >> 172.21.15.62:0/4124009788 conn(0x7fb53809e550 msgr2=0x7fb5380d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 -- 172.21.15.62:0/4124009788 shutdown_connections 2024-04-26T22:27:10.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.904+0000 7fb53d79d640 1 -- 172.21.15.62:0/4124009788 wait complete. 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.905+0000 7fb53d79d640 1 Processor -- start 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.905+0000 7fb53d79d640 1 -- start start 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5380d2320 0x7fb5381eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb538160800 0x7fb5381e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380b6300 con 0x7fb5380d5b60 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380d30f0 con 0x7fb5380d2320 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb53d79d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb538160be0 con 0x7fb538160800 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50916/0 (socket says 172.21.15.62:50916) 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 -- 172.21.15.62:0/3237386401 learned_addr learned my addr 172.21.15.62:0/3237386401 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb536ffd640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5380d2320 0x7fb5381eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5377fe640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb538160800 0x7fb5381e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 -- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb538160800 msgr2=0x7fb5381e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb538160800 0x7fb5381e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 -- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5380d2320 msgr2=0x7fb5381eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5380d2320 0x7fb5381eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 -- 172.21.15.62:0/3237386401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381eac10 con 0x7fb5380d5b60 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.906+0000 7fb5367fc640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c500 tx=0x7fb52000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.907+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520018070 con 0x7fb5380d5b60 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.907+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb520007580 con 0x7fb5380d5b60 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.907+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520004a20 con 0x7fb5380d5b60 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.907+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5380c9a40 con 0x7fb5380d5b60 2024-04-26T22:27:10.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.907+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381eaed0 con 0x7fb5380d5b60 2024-04-26T22:27:10.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.908+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53808e920 con 0x7fb5380d5b60 2024-04-26T22:27:10.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.908+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb520004bc0 con 0x7fb5380d5b60 2024-04-26T22:27:10.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.909+0000 7fb52f7fe640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 0x7fb51806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:10.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.909+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb52008aa30 con 0x7fb5380d5b60 2024-04-26T22:27:10.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.909+0000 7fb536ffd640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 0x7fb51806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:10.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.909+0000 7fb536ffd640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 0x7fb51806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb530034000 tx=0x7fb53002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:10.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:10.913+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb53808e920 con 0x7fb5380d5b60 2024-04-26T22:27:11.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.096+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb53806a360 con 0x7fb5380d5b60 2024-04-26T22:27:11.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.096+0000 7fb52f7fe640 1 -- 172.21.15.62:0/3237386401 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fb5200591c0 con 0x7fb5380d5b60 2024-04-26T22:27:11.097 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 msgr2=0x7fb51806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 0x7fb51806ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb530034000 tx=0x7fb53002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 msgr2=0x7fb53814fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c500 tx=0x7fb52000c9c0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 shutdown_connections 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb51806c920 0x7fb51806ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb538160800 0x7fb5381e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb5380d5b60 0x7fb53814fa30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 --2- 172.21.15.62:0/3237386401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5380d2320 0x7fb5381eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 >> 172.21.15.62:0/3237386401 conn(0x7fb53809e550 msgr2=0x7fb53806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 shutdown_connections 2024-04-26T22:27:11.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.098+0000 7fb53d79d640 1 -- 172.21.15.62:0/3237386401 wait complete. 2024-04-26T22:27:11.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T22:27:11.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.210+0000 7fabed90a640 1 Processor -- start 2024-04-26T22:27:11.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.210+0000 7fabed90a640 1 -- start start 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabed90a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabe806ea40 0x7fabe80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabed90a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabe80cf830 0x7fabe80cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabed90a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabe80d0180 con 0x7fabe8162e40 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabed90a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fabe8163220 con 0x7fabe806ea40 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabed90a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fabe80d0eb0 con 0x7fabe80cf830 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabe806ea40 0x7fabe80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabe806ea40 0x7fabe80d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50924/0 (socket says 172.21.15.62:50924) 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 -- 172.21.15.62:0/2381134506 learned_addr learned my addr 172.21.15.62:0/2381134506 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe77fe640 1 --2- 172.21.15.62:0/2381134506 >> v2:172.21.15.174:3300/0 conn(0x7fabe80cf830 0x7fabe80cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 -- 172.21.15.62:0/2381134506 >> v2:172.21.15.174:3300/0 conn(0x7fabe80cf830 msgr2=0x7fabe80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 --2- 172.21.15.62:0/2381134506 >> v2:172.21.15.174:3300/0 conn(0x7fabe80cf830 0x7fabe80cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 -- 172.21.15.62:0/2381134506 >> v1:172.21.15.174:6789/0 conn(0x7fabe8162e40 legacy=0x7fabe80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:11.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.211+0000 7fabe67fc640 1 -- 172.21.15.62:0/2381134506 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80b1d70 con 0x7fabe806ea40 2024-04-26T22:27:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabe67fc640 1 --2- 172.21.15.62:0/2381134506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe806ea40 0x7fabe80d1340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabe80737a0 tx=0x7fabdc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587b315a1ddeb35f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabe5ffb640 1 -- 172.21.15.62:0/2381134506 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc037070 con 0x7fabe806ea40 2024-04-26T22:27:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabe5ffb640 1 -- 172.21.15.62:0/2381134506 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabdc032440 con 0x7fabe806ea40 2024-04-26T22:27:11.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabe5ffb640 1 -- 172.21.15.62:0/2381134506 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc032810 con 0x7fabe806ea40 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 -- 172.21.15.62:0/2381134506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe806ea40 msgr2=0x7fabe80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 --2- 172.21.15.62:0/2381134506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe806ea40 0x7fabe80d1340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabe80737a0 tx=0x7fabdc02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 -- 172.21.15.62:0/2381134506 shutdown_connections 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 --2- 172.21.15.62:0/2381134506 >> v2:172.21.15.174:3300/0 conn(0x7fabe80cf830 0x7fabe80cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 --2- 172.21.15.62:0/2381134506 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe806ea40 0x7fabe80d1340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 -- 172.21.15.62:0/2381134506 >> 172.21.15.62:0/2381134506 conn(0x7fabe809ba50 msgr2=0x7fabe809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 -- 172.21.15.62:0/2381134506 shutdown_connections 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.212+0000 7fabed90a640 1 -- 172.21.15.62:0/2381134506 wait complete. 2024-04-26T22:27:11.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.213+0000 7fabed90a640 1 Processor -- start 2024-04-26T22:27:11.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 -- start start 2024-04-26T22:27:11.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe806ea40 0x7fabe80cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe80cf830 0x7fabe80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80d1f20 con 0x7fabe8162e40 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabe80d1c40 con 0x7fabe80cf830 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabed90a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabe80ce010 con 0x7fabe806ea40 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.214+0000 7fabe77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50936/0 (socket says 172.21.15.62:50936) 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 -- 172.21.15.62:0/37294857 learned_addr learned my addr 172.21.15.62:0/37294857 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe6ffd640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe806ea40 0x7fabe80cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe67fc640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe80cf830 0x7fabe80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 -- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe806ea40 msgr2=0x7fabe80cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe806ea40 0x7fabe80cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 -- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe80cf830 msgr2=0x7fabe80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe80cf830 0x7fabe80cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 -- 172.21.15.62:0/37294857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe80cb270 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabe77fe640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabe0007e40 tx=0x7fabe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0014620 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0019070 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0014a20 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe80cb550 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.215+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe80cbb10 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.216+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe806a360 con 0x7fabe8162e40 2024-04-26T22:27:11.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.217+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe0014bc0 con 0x7fabe8162e40 2024-04-26T22:27:11.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.217+0000 7fabcffff640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 0x7fabd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.217+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fabe008c000 con 0x7fabe8162e40 2024-04-26T22:27:11.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.217+0000 7fabe6ffd640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 0x7fabd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.218+0000 7fabe6ffd640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 0x7fabd406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006fd0 tx=0x7fabd8009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.221+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabe005a0a0 con 0x7fabe8162e40 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.407+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fabe80d2950 con 0x7fabe8162e40 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.408+0000 7fabcffff640 1 -- 172.21.15.62:0/37294857 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fabe005e3a0 con 0x7fabe8162e40 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-26T22:27:11.408 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:11.409 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 msgr2=0x7fabd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 0x7fabd406edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabd8006fd0 tx=0x7fabd8009060 comp rx=0 tx=0).stop 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 msgr2=0x7fabe8157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabe0007e40 tx=0x7fabe000e4b0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 shutdown_connections 2024-04-26T22:27:11.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd406c8f0 0x7fabd406edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabe8162e40 0x7fabe8157590 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe80cf830 0x7fabe80cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 --2- 172.21.15.62:0/37294857 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabe806ea40 0x7fabe80cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 >> 172.21.15.62:0/37294857 conn(0x7fabe809ba50 msgr2=0x7fabe80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 shutdown_connections 2024-04-26T22:27:11.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.410+0000 7fabed90a640 1 -- 172.21.15.62:0/37294857 wait complete. 2024-04-26T22:27:11.420 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T22:27:11.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T22:27:11.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.524+0000 7fdd554bd640 1 Processor -- start 2024-04-26T22:27:11.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.524+0000 7fdd554bd640 1 -- start start 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd554bd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd5006c240 0x7fdd5006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd554bd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 0x7fdd500cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd554bd640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd500d0400 con 0x7fdd50145a20 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd554bd640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fdd500cdb70 con 0x7fdd5006c240 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd554bd640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fdd500ce710 con 0x7fdd500cfa90 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd4e7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd5006c240 0x7fdd5006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd4e7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdd5006c240 0x7fdd5006c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50942/0 (socket says 172.21.15.62:50942) 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/1515863132 learned_addr learned my addr 172.21.15.62:0/1515863132 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.525+0000 7fdd4effd640 1 --1- 172.21.15.62:0/1515863132 >> v1:172.21.15.62:6789/0 conn(0x7fdd50145a20 0x7fdd500ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57714/0 (socket says 172.21.15.62:57714) 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4f7fe640 1 --2- 172.21.15.62:0/1515863132 >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 0x7fdd500cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/1515863132 >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 msgr2=0x7fdd500cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4e7fc640 1 --2- 172.21.15.62:0/1515863132 >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 0x7fdd500cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/1515863132 >> v1:172.21.15.62:6789/0 conn(0x7fdd50145a20 legacy=0x7fdd500ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4dffb640 1 -- 172.21.15.62:0/1515863132 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 774527107 0 0) 0x7fdd500d0400 con 0x7fdd50145a20 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/1515863132 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd50145e00 con 0x7fdd5006c240 2024-04-26T22:27:11.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4f7fe640 1 --2- 172.21.15.62:0/1515863132 >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 0x7fdd500cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:11.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.526+0000 7fdd4e7fc640 1 --2- 172.21.15.62:0/1515863132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5006c240 0x7fdd5006c6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd500b1a90 tx=0x7fdd3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36b241c70766f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd4dffb640 1 -- 172.21.15.62:0/1515863132 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38037070 con 0x7fdd5006c240 2024-04-26T22:27:11.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd4dffb640 1 -- 172.21.15.62:0/1515863132 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38032440 con 0x7fdd5006c240 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd4dffb640 1 -- 172.21.15.62:0/1515863132 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38032810 con 0x7fdd5006c240 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 -- 172.21.15.62:0/1515863132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5006c240 msgr2=0x7fdd5006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 --2- 172.21.15.62:0/1515863132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5006c240 0x7fdd5006c6c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd500b1a90 tx=0x7fdd3802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 -- 172.21.15.62:0/1515863132 shutdown_connections 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 --2- 172.21.15.62:0/1515863132 >> v2:172.21.15.174:3300/0 conn(0x7fdd500cfa90 0x7fdd500cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 --2- 172.21.15.62:0/1515863132 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd5006c240 0x7fdd5006c6c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 -- 172.21.15.62:0/1515863132 >> 172.21.15.62:0/1515863132 conn(0x7fdd500d1ec0 msgr2=0x7fdd500d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 -- 172.21.15.62:0/1515863132 shutdown_connections 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.527+0000 7fdd554bd640 1 -- 172.21.15.62:0/1515863132 wait complete. 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.528+0000 7fdd554bd640 1 Processor -- start 2024-04-26T22:27:11.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.528+0000 7fdd554bd640 1 -- start start 2024-04-26T22:27:11.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd5006c240 0x7fdd501343a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 0x7fdd500cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdd500cd800 con 0x7fdd500cfa90 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd500cc190 con 0x7fdd5006c240 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd554bd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdd500cd4b0 con 0x7fdd50145a20 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd4e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd4e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50948/0 (socket says 172.21.15.62:50948) 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/3655830987 learned_addr learned my addr 172.21.15.62:0/3655830987 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd4effd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd5006c240 0x7fdd501343a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.529+0000 7fdd4f7fe640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 0x7fdd500cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 msgr2=0x7fdd500cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 0x7fdd500cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd5006c240 msgr2=0x7fdd501343a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd5006c240 0x7fdd501343a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 -- 172.21.15.62:0/3655830987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd501352f0 con 0x7fdd500cfa90 2024-04-26T22:27:11.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4f7fe640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 0x7fdd500cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd4e7fc640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd3802a220 tx=0x7fdd380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38037040 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38042070 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd50134a50 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38030bb0 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.530+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd50135010 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.531+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd500693e0 con 0x7fdd500cfa90 2024-04-26T22:27:11.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.531+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd38047460 con 0x7fdd500cfa90 2024-04-26T22:27:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.532+0000 7fdd477fe640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 0x7fdd3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.532+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdd380ac390 con 0x7fdd500cfa90 2024-04-26T22:27:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.532+0000 7fdd4effd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 0x7fdd3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.532+0000 7fdd4effd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 0x7fdd3006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.536+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd3807ab10 con 0x7fdd500cfa90 2024-04-26T22:27:11.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.721+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fdd500d2dd0 con 0x7fdd500cfa90 2024-04-26T22:27:11.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.842+0000 7fdd477fe640 1 -- 172.21.15.62:0/3655830987 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fdd3807a410 con 0x7fdd500cfa90 2024-04-26T22:27:11.842 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-26T22:27:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 msgr2=0x7fdd3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 0x7fdd3006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48007d50 comp rx=0 tx=0).stop 2024-04-26T22:27:11.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 msgr2=0x7fdd50132350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd3802a220 tx=0x7fdd380046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 shutdown_connections 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdd3006c8f0 0x7fdd3006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdd50145a20 0x7fdd500cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdd500cfa90 0x7fdd50132350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 --2- 172.21.15.62:0/3655830987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd5006c240 0x7fdd501343a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 >> 172.21.15.62:0/3655830987 conn(0x7fdd500d1ec0 msgr2=0x7fdd500d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 shutdown_connections 2024-04-26T22:27:11.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.844+0000 7fdd554bd640 1 -- 172.21.15.62:0/3655830987 wait complete. 2024-04-26T22:27:11.859 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T22:27:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.966+0000 7f0caf27b640 1 Processor -- start 2024-04-26T22:27:11.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.966+0000 7f0caf27b640 1 -- start start 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.966+0000 7f0caf27b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ca815f630 0x7f0ca80d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0caf27b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ca80d5ca0 0x7f0ca80d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0caf27b640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca80d0610 con 0x7f0ca80d20b0 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0caf27b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca80d24e0 con 0x7f0ca815f630 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0caf27b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca80d08f0 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ca815f630 0x7f0ca80d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0ca815f630 0x7f0ca80d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50954/0 (socket says 172.21.15.62:50954) 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad818640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0ca80d20b0 0x7f0ca81631f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57722/0 (socket says 172.21.15.62:57722) 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 -- 172.21.15.62:0/3442935246 learned_addr learned my addr 172.21.15.62:0/3442935246 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 -- 172.21.15.62:0/3442935246 >> v2:172.21.15.174:3300/0 conn(0x7f0ca80d5ca0 msgr2=0x7f0ca80d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:11.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cac816640 1 --2- 172.21.15.62:0/3442935246 >> v2:172.21.15.174:3300/0 conn(0x7f0ca80d5ca0 0x7f0ca80d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 --2- 172.21.15.62:0/3442935246 >> v2:172.21.15.174:3300/0 conn(0x7f0ca80d5ca0 0x7f0ca80d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 -- 172.21.15.62:0/3442935246 >> v1:172.21.15.62:6789/0 conn(0x7f0ca80d20b0 legacy=0x7f0ca81631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0cad017640 1 -- 172.21.15.62:0/3442935246 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80d0bd0 con 0x7f0ca815f630 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.967+0000 7f0c9ffff640 1 -- 172.21.15.62:0/3442935246 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1355427437 0 0) 0x7f0ca80d0610 con 0x7f0ca80d20b0 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0cad017640 1 --2- 172.21.15.62:0/3442935246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca815f630 0x7f0ca80d0030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80d2a80 tx=0x7f0ca402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec14b1f0d54ad0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0c9ffff640 1 -- 172.21.15.62:0/3442935246 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4037070 con 0x7f0ca815f630 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0c9ffff640 1 -- 172.21.15.62:0/3442935246 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca4032440 con 0x7f0ca815f630 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0c9ffff640 1 -- 172.21.15.62:0/3442935246 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4032810 con 0x7f0ca815f630 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 -- 172.21.15.62:0/3442935246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca815f630 msgr2=0x7f0ca80d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 --2- 172.21.15.62:0/3442935246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca815f630 0x7f0ca80d0030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80d2a80 tx=0x7f0ca402a270 comp rx=0 tx=0).stop 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 -- 172.21.15.62:0/3442935246 shutdown_connections 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 --2- 172.21.15.62:0/3442935246 >> v2:172.21.15.174:3300/0 conn(0x7f0ca80d5ca0 0x7f0ca80d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 --2- 172.21.15.62:0/3442935246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca815f630 0x7f0ca80d0030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 -- 172.21.15.62:0/3442935246 >> 172.21.15.62:0/3442935246 conn(0x7f0ca809e530 msgr2=0x7f0ca80d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:11.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 -- 172.21.15.62:0/3442935246 shutdown_connections 2024-04-26T22:27:11.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.968+0000 7f0caf27b640 1 -- 172.21.15.62:0/3442935246 wait complete. 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 Processor -- start 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 -- start start 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ca80d20b0 0x7f0ca80b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ca815f630 0x7f0ca815d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca815fa10 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d1190 con 0x7f0ca80d20b0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0caf27b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80d3180 con 0x7f0ca815f630 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0cac816640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0cac816640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:50960/0 (socket says 172.21.15.62:50960) 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0cac816640 1 -- 172.21.15.62:0/2873911529 learned_addr learned my addr 172.21.15.62:0/2873911529 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.970+0000 7f0cad818640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ca815f630 0x7f0ca815d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cad017640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ca80d20b0 0x7f0ca80b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 -- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ca815f630 msgr2=0x7f0ca815d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ca815f630 0x7f0ca815d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 -- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ca80d20b0 msgr2=0x7f0ca80b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ca80d20b0 0x7f0ca80b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 -- 172.21.15.62:0/2873911529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80b85a0 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0cac816640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c820 tx=0x7f0c9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98015070 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c98004240 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800ce10 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca80b88e0 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.971+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca80b9120 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.972+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca808d690 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.973+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c98007830 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.973+0000 7f0c9dffb640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 0x7f0c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.973+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f0c9808b750 con 0x7f0ca80d5ca0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.973+0000 7f0cad017640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 0x7f0c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:11.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.974+0000 7f0cad017640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 0x7f0c8c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4034000 tx=0x7f0ca402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:11.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:11.978+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ca808d690 con 0x7f0ca80d5ca0 2024-04-26T22:27:12.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.164+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ca80b8ac0 con 0x7f0ca80d5ca0 2024-04-26T22:27:12.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.844+0000 7f0c9dffb640 1 -- 172.21.15.62:0/2873911529 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f0ca80b88e0 con 0x7f0ca80d5ca0 2024-04-26T22:27:12.845 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-26T22:27:12.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 msgr2=0x7f0c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:12.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 0x7f0c8c06edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4034000 tx=0x7f0ca402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 msgr2=0x7f0ca80b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0c9800c820 tx=0x7f0c9800cce0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 shutdown_connections 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0c8c06c8f0 0x7f0c8c06edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ca815f630 0x7f0ca815d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0ca80d5ca0 0x7f0ca80b33c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 --2- 172.21.15.62:0/2873911529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ca80d20b0 0x7f0ca80b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 >> 172.21.15.62:0/2873911529 conn(0x7f0ca809e530 msgr2=0x7f0ca80b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 shutdown_connections 2024-04-26T22:27:12.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.847+0000 7f0caf27b640 1 -- 172.21.15.62:0/2873911529 wait complete. 2024-04-26T22:27:12.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T22:27:12.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.982+0000 7fa2c5c0f640 1 Processor -- start 2024-04-26T22:27:12.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.982+0000 7fa2c5c0f640 1 -- start start 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2c5c0f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2c0160800 0x7fa2c01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2c5c0f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 0x7fa2c00d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2c5c0f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c00d0930 con 0x7fa2c00d5b60 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2c5c0f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00d0c00 con 0x7fa2c0160800 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2c5c0f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00d0ee0 con 0x7fa2c00d2320 2024-04-26T22:27:12.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2c0160800 0x7fa2c01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa2c0160800 0x7fa2c01610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35196/0 (socket says 172.21.15.62:35196) 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 -- 172.21.15.62:0/3785510104 learned_addr learned my addr 172.21.15.62:0/3785510104 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bffff640 1 --2- 172.21.15.62:0/3785510104 >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 0x7fa2c00d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 -- 172.21.15.62:0/3785510104 >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 msgr2=0x7fa2c00d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 --2- 172.21.15.62:0/3785510104 >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 0x7fa2c00d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 -- 172.21.15.62:0/3785510104 >> v1:172.21.15.174:6789/0 conn(0x7fa2c00d5b60 legacy=0x7fa2c00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bf7fe640 1 -- 172.21.15.62:0/3785510104 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00d11c0 con 0x7fa2c0160800 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.983+0000 7fa2bffff640 1 --2- 172.21.15.62:0/3785510104 >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 0x7fa2c00d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.984+0000 7fa2bf7fe640 1 --2- 172.21.15.62:0/3785510104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c01610c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d4750 tx=0x7fa2b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad2984f84d112a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.984+0000 7fa2be7fc640 1 -- 172.21.15.62:0/3785510104 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4037070 con 0x7fa2c0160800 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.984+0000 7fa2be7fc640 1 -- 172.21.15.62:0/3785510104 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b4032440 con 0x7fa2c0160800 2024-04-26T22:27:12.985 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.984+0000 7fa2be7fc640 1 -- 172.21.15.62:0/3785510104 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4032810 con 0x7fa2c0160800 2024-04-26T22:27:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/3785510104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 msgr2=0x7fa2c01610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/3785510104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c01610c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00d4750 tx=0x7fa2b402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/3785510104 shutdown_connections 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/3785510104 >> v2:172.21.15.174:3300/0 conn(0x7fa2c00d2320 0x7fa2c00d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/3785510104 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c01610c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/3785510104 >> 172.21.15.62:0/3785510104 conn(0x7fa2c009e550 msgr2=0x7fa2c00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/3785510104 shutdown_connections 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.985+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/3785510104 wait complete. 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.986+0000 7fa2c5c0f640 1 Processor -- start 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.986+0000 7fa2c5c0f640 1 -- start start 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00d2320 0x7fa2c0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00d5b60 0x7fa2c00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00b6300 con 0x7fa2c0160800 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00d30f0 con 0x7fa2c00d2320 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c0160be0 con 0x7fa2c00d5b60 2024-04-26T22:27:12.989 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35210/0 (socket says 172.21.15.62:35210) 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 -- 172.21.15.62:0/1060460068 learned_addr learned my addr 172.21.15.62:0/1060460068 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2beffd640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00d5b60 0x7fa2c00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bf7fe640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00d2320 0x7fa2c0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 -- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00d5b60 msgr2=0x7fa2c00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00d5b60 0x7fa2c00b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 -- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00d2320 msgr2=0x7fa2c0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00d2320 0x7fa2c0154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 -- 172.21.15.62:0/1060460068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00b82d0 con 0x7fa2c0160800 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2bffff640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ce70 tx=0x7fa2b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b801a070 con 0x7fa2c0160800 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b800d840 con 0x7fa2c0160800 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c00b85b0 con 0x7fa2c0160800 2024-04-26T22:27:12.990 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800dc30 con 0x7fa2c0160800 2024-04-26T22:27:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.987+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c00b8b70 con 0x7fa2c0160800 2024-04-26T22:27:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.989+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c006a360 con 0x7fa2c0160800 2024-04-26T22:27:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.989+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b801d070 con 0x7fa2c0160800 2024-04-26T22:27:12.991 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.989+0000 7fa2a7fff640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 0x7fa2ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:12.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.989+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7fa2b808c290 con 0x7fa2c0160800 2024-04-26T22:27:12.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.990+0000 7fa2bf7fe640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 0x7fa2ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:12.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.990+0000 7fa2bf7fe640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 0x7fa2ac06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4034000 tx=0x7fa2b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:12.994 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:12.994+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2b805a7b0 con 0x7fa2c0160800 2024-04-26T22:27:13.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.220+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2c00d2ba0 con 0x7fa2c0160800 2024-04-26T22:27:13.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:27:13.518 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:27:13.869 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.869+0000 7fa2a7fff640 1 -- 172.21.15.62:0/1060460068 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa2b805a0b0 con 0x7fa2c0160800 2024-04-26T22:27:13.870 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T22:27:13.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 msgr2=0x7fa2ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 0x7fa2ac06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4034000 tx=0x7fa2b402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 msgr2=0x7fa2c00b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800ce70 tx=0x7fa2b800a710 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 shutdown_connections 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2ac06c8f0 0x7fa2ac06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2c0160800 0x7fa2c00b3660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2c00d5b60 0x7fa2c00b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 --2- 172.21.15.62:0/1060460068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2c00d2320 0x7fa2c0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:13.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.877+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 >> 172.21.15.62:0/1060460068 conn(0x7fa2c009e550 msgr2=0x7fa2c0162490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.878+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 shutdown_connections 2024-04-26T22:27:13.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.878+0000 7fa2c5c0f640 1 -- 172.21.15.62:0/1060460068 wait complete. 2024-04-26T22:27:13.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:13.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.996+0000 7fa4cc612640 1 Processor -- start 2024-04-26T22:27:13.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.996+0000 7fa4cc612640 1 -- start start 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cc612640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 0x7fa4c40d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cc612640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa4c40d3c80 0x7fa4c40d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cc612640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c40d0210 con 0x7fa4c40d3820 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cc612640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c40d04f0 con 0x7fa4c40d3c80 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cc612640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c40d07d0 con 0x7fa4c4162b40 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.997+0000 7fa4cabaf640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa4c40d3c80 0x7fa4c40d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4c9bad640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa4c40d3820 0x7fa4c40d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56308/0 (socket says 172.21.15.62:56308) 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4c9bad640 1 -- 172.21.15.62:0/2132595098 learned_addr learned my addr 172.21.15.62:0/2132595098 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:13.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 0x7fa4c40d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4cabaf640 1 -- 172.21.15.62:0/2132595098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 msgr2=0x7fa4c40d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4cabaf640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 0x7fa4c40d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4cabaf640 1 -- 172.21.15.62:0/2132595098 >> v1:172.21.15.62:6789/0 conn(0x7fa4c40d3820 legacy=0x7fa4c40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4c93ac640 1 -- 172.21.15.62:0/2132595098 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973560547 0 0) 0x7fa4c40d0210 con 0x7fa4c40d3820 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4cabaf640 1 -- 172.21.15.62:0/2132595098 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40d0ab0 con 0x7fa4c40d3c80 2024-04-26T22:27:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.998+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 0x7fa4c40d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cabaf640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40d1f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009d90 tx=0x7fa4bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc688adc756fbbb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4c93ac640 1 -- 172.21.15.62:0/2132595098 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc007740 con 0x7fa4c40d3c80 2024-04-26T22:27:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4c93ac640 1 -- 172.21.15.62:0/2132595098 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0078e0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4c93ac640 1 -- 172.21.15.62:0/2132595098 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc007cb0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 -- 172.21.15.62:0/2132595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 msgr2=0x7fa4c40d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40d1f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009d90 tx=0x7fa4bc02a6f0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 -- 172.21.15.62:0/2132595098 shutdown_connections 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40d1f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 --2- 172.21.15.62:0/2132595098 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c4162b40 0x7fa4c40d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 -- 172.21.15.62:0/2132595098 >> 172.21.15.62:0/2132595098 conn(0x7fa4c409e550 msgr2=0x7fa4c40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 -- 172.21.15.62:0/2132595098 shutdown_connections 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:13.999+0000 7fa4cc612640 1 -- 172.21.15.62:0/2132595098 wait complete. 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.000+0000 7fa4cc612640 1 Processor -- start 2024-04-26T22:27:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.000+0000 7fa4cc612640 1 -- start start 2024-04-26T22:27:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 0x7fa4c414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4c4162b40 0x7fa4c40c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40b5bb0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c40d4110 con 0x7fa4c4162b40 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4cc612640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c40d6110 con 0x7fa4c40d3820 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4c9bad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4c9bad640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35232/0 (socket says 172.21.15.62:35232) 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4c9bad640 1 -- 172.21.15.62:0/1196929969 learned_addr learned my addr 172.21.15.62:0/1196929969 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.001+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 0x7fa4c414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4cabaf640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4c4162b40 0x7fa4c40c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 -- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 msgr2=0x7fa4c414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 0x7fa4c414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 -- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4c4162b40 msgr2=0x7fa4c40c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4c4162b40 0x7fa4c40c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 -- 172.21.15.62:0/1196929969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c40ca9b0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 0x7fa4c414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4c9bad640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c430 tx=0x7fa4b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40074e0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b40047e0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c40cac90 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4004bd0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.002+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c40cb250 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.003+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c406a360 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.003+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4b4011070 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.004+0000 7fa4baffd640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 0x7fa4a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.004+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b408b7e0 con 0x7fa4c40d3c80 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.004+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 0x7fa4a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.004+0000 7fa4ca3ae640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 0x7fa4a806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0006fd0 tx=0x7fa4c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.008+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4b4059c10 con 0x7fa4c40d3c80 2024-04-26T22:27:14.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.193+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4c40d2960 con 0x7fa4c40d3c80 2024-04-26T22:27:14.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.194+0000 7fa4baffd640 1 -- 172.21.15.62:0/1196929969 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7fa4b4059510 con 0x7fa4c40d3c80 2024-04-26T22:27:14.195 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:14.195 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:13.221778+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 msgr2=0x7fa4a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 0x7fa4a806edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0006fd0 tx=0x7fa4c0008040 comp rx=0 tx=0).stop 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 msgr2=0x7fa4c40b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400c430 tx=0x7fa4b400c8f0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 shutdown_connections 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa4a806c8f0 0x7fa4a806edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4c4162b40 0x7fa4c40c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4c40d3c80 0x7fa4c40b8530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 --2- 172.21.15.62:0/1196929969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa4c40d3820 0x7fa4c414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 >> 172.21.15.62:0/1196929969 conn(0x7fa4c409e550 msgr2=0x7fa4c40d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:14.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 shutdown_connections 2024-04-26T22:27:14.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.196+0000 7fa4cc612640 1 -- 172.21.15.62:0/1196929969 wait complete. 2024-04-26T22:27:14.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.307+0000 7f24376b8640 1 Processor -- start 2024-04-26T22:27:14.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.307+0000 7f24376b8640 1 -- start start 2024-04-26T22:27:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f24376b8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 0x7f243006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f24376b8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2430160320 0x7f24300ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f24376b8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2430072e30 con 0x7f2430070130 2024-04-26T22:27:14.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f24376b8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2430073110 con 0x7f2430160320 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f24376b8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f24300733f0 con 0x7f2430159a90 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f2435c55640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2430160320 0x7f24300ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f2435c55640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2430160320 0x7f24300ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35248/0 (socket says 172.21.15.62:35248) 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f2435c55640 1 -- 172.21.15.62:0/2425177053 learned_addr learned my addr 172.21.15.62:0/2425177053 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.308+0000 7f2435454640 1 --2- 172.21.15.62:0/2425177053 >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 0x7f243006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435c55640 1 -- 172.21.15.62:0/2425177053 >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 msgr2=0x7f243006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435c55640 1 --2- 172.21.15.62:0/2425177053 >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 0x7f243006fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435c55640 1 -- 172.21.15.62:0/2425177053 >> v1:172.21.15.174:6789/0 conn(0x7f2430070130 legacy=0x7f243015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435c55640 1 -- 172.21.15.62:0/2425177053 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24300736d0 con 0x7f2430160320 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435454640 1 --2- 172.21.15.62:0/2425177053 >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 0x7f243006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2435c55640 1 --2- 172.21.15.62:0/2425177053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ce9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f242c009e50 tx=0x7f242c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401c894c2282dc54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2427fff640 1 -- 172.21.15.62:0/2425177053 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c036070 con 0x7f2430160320 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2427fff640 1 -- 172.21.15.62:0/2425177053 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c031600 con 0x7f2430160320 2024-04-26T22:27:14.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.309+0000 7f2427fff640 1 -- 172.21.15.62:0/2425177053 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c0319d0 con 0x7f2430160320 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 -- 172.21.15.62:0/2425177053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 msgr2=0x7f24300ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 --2- 172.21.15.62:0/2425177053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ce9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f242c009e50 tx=0x7f242c02a640 comp rx=0 tx=0).stop 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 -- 172.21.15.62:0/2425177053 shutdown_connections 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 --2- 172.21.15.62:0/2425177053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ce9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 --2- 172.21.15.62:0/2425177053 >> v2:172.21.15.174:3300/0 conn(0x7f2430159a90 0x7f243006fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 -- 172.21.15.62:0/2425177053 >> 172.21.15.62:0/2425177053 conn(0x7f24300d0d50 msgr2=0x7f24300d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 -- 172.21.15.62:0/2425177053 shutdown_connections 2024-04-26T22:27:14.312 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.310+0000 7f24376b8640 1 -- 172.21.15.62:0/2425177053 wait complete. 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.311+0000 7f24376b8640 1 Processor -- start 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.311+0000 7f24376b8640 1 -- start start 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2430070130 0x7f24300b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2430159a90 0x7f24300b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2430159ff0 con 0x7f2430160320 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2430159e70 con 0x7f2430159a90 2024-04-26T22:27:14.315 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f24376b8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f24300ceef0 con 0x7f2430070130 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f2435c55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f2435454640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2430070130 0x7f24300b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f2435c55640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35258/0 (socket says 172.21.15.62:35258) 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f2435c55640 1 -- 172.21.15.62:0/3643552666 learned_addr learned my addr 172.21.15.62:0/3643552666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.312+0000 7f2434c53640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2430159a90 0x7f24300b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 -- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2430070130 msgr2=0x7f24300b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2430070130 0x7f24300b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 -- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2430159a90 msgr2=0x7f24300b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2430159a90 0x7f24300b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 -- 172.21.15.62:0/3643552666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243015da40 con 0x7f2430160320 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2435c55640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f242c034040 tx=0x7f242c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c036050 con 0x7f2430160320 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242c042070 con 0x7f2430160320 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243015dd20 con 0x7f2430160320 2024-04-26T22:27:14.316 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c009510 con 0x7f2430160320 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.313+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243015e2e0 con 0x7f2430160320 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.314+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243006a360 con 0x7f2430160320 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.315+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f242c007bd0 con 0x7f2430160320 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.315+0000 7f2425ffb640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.315+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f242c0ac7e0 con 0x7f2430160320 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.315+0000 7f2435454640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.316+0000 7f2435454640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 0x7f241406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2428006fd0 tx=0x7f2428008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.319+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f242c02ae20 con 0x7f2430160320 2024-04-26T22:27:14.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.504+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24300d0730 con 0x7f2430160320 2024-04-26T22:27:14.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.505+0000 7f2425ffb640 1 -- 172.21.15.62:0/3643552666 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7f242c07a650 con 0x7f2430160320 2024-04-26T22:27:14.505 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:14.505 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:13.221778+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:14.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 msgr2=0x7f241406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 0x7f241406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2428006fd0 tx=0x7f2428008040 comp rx=0 tx=0).stop 2024-04-26T22:27:14.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 msgr2=0x7f24300ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f242c034040 tx=0x7f242c004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 shutdown_connections 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f241406c8f0 0x7f241406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2430160320 0x7f24300ca5d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2430159a90 0x7f24300b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 --2- 172.21.15.62:0/3643552666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2430070130 0x7f24300b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 >> 172.21.15.62:0/3643552666 conn(0x7f24300d0d50 msgr2=0x7f2430157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 shutdown_connections 2024-04-26T22:27:14.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.507+0000 7f24376b8640 1 -- 172.21.15.62:0/3643552666 wait complete. 2024-04-26T22:27:14.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T22:27:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.622+0000 7f1790037640 1 Processor -- start 2024-04-26T22:27:14.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.622+0000 7f1790037640 1 -- start start 2024-04-26T22:27:14.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f1790037640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1788160800 0x7f17881610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f1790037640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f17880d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f1790037640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d0930 con 0x7f17880d5b60 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f1790037640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d0c00 con 0x7f1788160800 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f1790037640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f17880d0ee0 con 0x7f17880d2320 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178d5d2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f17880d5b60 0x7f17880d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56318/0 (socket says 172.21.15.62:56318) 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178d5d2640 1 -- 172.21.15.62:0/2477600468 learned_addr learned my addr 172.21.15.62:0/2477600468 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f17880d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2477600468 >> v2:172.21.15.174:3300/0 conn(0x7f1788160800 0x7f17881610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794888335 0 0) 0x7f17880d0930 con 0x7f17880d5b60 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1774003520 con 0x7f17880d5b60 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4085524506 0 0) 0x7f1774003520 con 0x7f17880d5b60 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 msgr2=0x7f17880d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:14.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f17880d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 >> v2:172.21.15.174:3300/0 conn(0x7f1788160800 msgr2=0x7f17881610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 --2- 172.21.15.62:0/2477600468 >> v2:172.21.15.174:3300/0 conn(0x7f1788160800 0x7f17881610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880d11c0 con 0x7f17880d5b60 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f1778001670 con 0x7f17880d5b60 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 >> v1:172.21.15.62:6789/0 conn(0x7f17880d5b60 legacy=0x7f17880d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f1774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1774004d40 0x7f17740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d0c00 con 0x7f1774004d40 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d0ee0 con 0x7f17740039b0 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.623+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f177400b9c0 con 0x7f17880d2320 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1774004d40 0x7f17740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178d5d2640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f1774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 -- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 msgr2=0x7f1774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f17880d2320 0x7f1774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 -- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17740039b0 msgr2=0x7f1774003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17740039b0 0x7f1774003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 -- 172.21.15.62:0/2477600468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1774003520 con 0x7f1774004d40 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.624+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1774004d40 0x7f17740051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1780009fc0 tx=0x7f1774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d78dc64bdbf238 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780009c10 con 0x7f1774004d40 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780036070 con 0x7f1774004d40 2024-04-26T22:27:14.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f178cdd1640 1 -- 172.21.15.62:0/2477600468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780031880 con 0x7f1774004d40 2024-04-26T22:27:14.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 -- 172.21.15.62:0/2477600468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1774004d40 msgr2=0x7f17740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 --2- 172.21.15.62:0/2477600468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1774004d40 0x7f17740051c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1780009fc0 tx=0x7f1774005830 comp rx=0 tx=0).stop 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f178e5d4640 1 -- 172.21.15.62:0/2477600468 reap_dead start 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 -- 172.21.15.62:0/2477600468 shutdown_connections 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 -- 172.21.15.62:0/2477600468 >> 172.21.15.62:0/2477600468 conn(0x7f178809e550 msgr2=0x7f17880d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 -- 172.21.15.62:0/2477600468 shutdown_connections 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.625+0000 7f1790037640 1 -- 172.21.15.62:0/2477600468 wait complete. 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.626+0000 7f1790037640 1 Processor -- start 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 -- start start 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 0x7f1788158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17881471e0 0x7f1788147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f17880b6300 con 0x7f1788147b80 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d30f0 con 0x7f17881471e0 2024-04-26T22:27:14.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f1790037640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1788160be0 con 0x7f1788146e00 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f178e5d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f178e5d4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35272/0 (socket says 172.21.15.62:35272) 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f178e5d4640 1 -- 172.21.15.62:0/2872585569 learned_addr learned my addr 172.21.15.62:0/2872585569 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f178d5d2640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17881471e0 0x7f1788147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.627+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 0x7f1788158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 -- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 msgr2=0x7f1788158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 0x7f1788158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 -- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17881471e0 msgr2=0x7f1788147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17881471e0 0x7f1788147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 -- 172.21.15.62:0/2872585569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880b9530 con 0x7f1788147b80 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 0x7f1788158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f178e5d4640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1780033040 tx=0x7f1780009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780009c10 con 0x7f1788147b80 2024-04-26T22:27:14.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780031d50 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880b9810 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780003bd0 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.628+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17880b9dd0 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.629+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178806a360 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.629+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1780005d10 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.630+0000 7f177e7fc640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.630+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f17800acd40 con 0x7f1788147b80 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.630+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:14.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.630+0000 7f178ddd3640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 0x7f176c06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1784009bc0 tx=0x7f1784008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:14.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.634+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f178004a060 con 0x7f1788147b80 2024-04-26T22:27:14.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:14.862+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f17880b9fa0 con 0x7f1788147b80 2024-04-26T22:27:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.860+0000 7f177e7fc640 1 -- 172.21.15.62:0/2872585569 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7f178007b170 con 0x7f1788147b80 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.863+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 msgr2=0x7f176c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.863+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 0x7f176c06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1784009bc0 tx=0x7f1784008040 comp rx=0 tx=0).stop 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.863+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 msgr2=0x7f1788145420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.863+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1780033040 tx=0x7f1780009750 comp rx=0 tx=0).stop 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 shutdown_connections 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f176c06c8f0 0x7f176c06edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1788147b80 0x7f1788145420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17881471e0 0x7f1788147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:15.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 --2- 172.21.15.62:0/2872585569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1788146e00 0x7f1788158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:15.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 >> 172.21.15.62:0/2872585569 conn(0x7f178809e550 msgr2=0x7f17881541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:15.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 shutdown_connections 2024-04-26T22:27:15.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.864+0000 7f1790037640 1 -- 172.21.15.62:0/2872585569 wait complete. 2024-04-26T22:27:15.878 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T22:27:15.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T22:27:15.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.992+0000 7f8aca625640 1 Processor -- start 2024-04-26T22:27:15.993 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.992+0000 7f8aca625640 1 -- start start 2024-04-26T22:27:15.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8aca625640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 0x7f8ac40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8aca625640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac40b2760 con 0x7f8ac415f400 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8aca625640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac40b2a00 con 0x7f8ac40b4b20 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8aca625640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40b43e0 con 0x7f8ac40d5a80 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8ac3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8ac415f400 0x7f8ac40b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56334/0 (socket says 172.21.15.62:56334) 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8ac3fff640 1 -- 172.21.15.62:0/4046024633 learned_addr learned my addr 172.21.15.62:0/4046024633 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.993+0000 7f8ac37fe640 1 --2- 172.21.15.62:0/4046024633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 0x7f8ac40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4252826699 0 0) 0x7f8ac40b2760 con 0x7f8ac415f400 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ab0003520 con 0x7f8ac415f400 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355775671 0 0) 0x7f8ac40b2a00 con 0x7f8ac40b4b20 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ac40b2760 con 0x7f8ac40b4b20 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac37fe640 1 -- 172.21.15.62:0/4046024633 >> v1:172.21.15.174:6789/0 conn(0x7f8ac40b4b20 legacy=0x7f8ac41630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:15.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac37fe640 1 -- 172.21.15.62:0/4046024633 >> v1:172.21.15.62:6789/0 conn(0x7f8ac415f400 legacy=0x7f8ac40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.994+0000 7f8ac37fe640 1 -- 172.21.15.62:0/4046024633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40b2ce0 con 0x7f8ac40d5a80 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8ac37fe640 1 --2- 172.21.15.62:0/4046024633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 0x7f8ac40b45e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097d0 tx=0x7f8ab402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29497a8488efe573 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4037070 con 0x7f8ac40d5a80 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4032440 con 0x7f8ac40d5a80 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8ac2ffd640 1 -- 172.21.15.62:0/4046024633 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40327f0 con 0x7f8ac40d5a80 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 -- 172.21.15.62:0/4046024633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 msgr2=0x7f8ac40b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 --2- 172.21.15.62:0/4046024633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 0x7f8ac40b45e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097d0 tx=0x7f8ab402a210 comp rx=0 tx=0).stop 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 -- 172.21.15.62:0/4046024633 shutdown_connections 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 --2- 172.21.15.62:0/4046024633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40d5a80 0x7f8ac40b45e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 -- 172.21.15.62:0/4046024633 >> 172.21.15.62:0/4046024633 conn(0x7f8ac409e4f0 msgr2=0x7f8ac40d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 -- 172.21.15.62:0/4046024633 shutdown_connections 2024-04-26T22:27:15.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.995+0000 7f8aca625640 1 -- 172.21.15.62:0/4046024633 wait complete. 2024-04-26T22:27:15.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.997+0000 7f8aca625640 1 Processor -- start 2024-04-26T22:27:15.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.997+0000 7f8aca625640 1 -- start start 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40b4b20 0x7f8ac41f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ac40d5a80 0x7f8ac415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac40b32a0 con 0x7f8ac415f400 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac40b3ed0 con 0x7f8ac40d5a80 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40b4f70 con 0x7f8ac40b4b20 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35280/0 (socket says 172.21.15.62:35280) 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 -- 172.21.15.62:0/2395190577 learned_addr learned my addr 172.21.15.62:0/2395190577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac37fe640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ac40d5a80 0x7f8ac415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac3fff640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40b4b20 0x7f8ac41f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 -- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40b4b20 msgr2=0x7f8ac41f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40b4b20 0x7f8ac41f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 -- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ac40d5a80 msgr2=0x7f8ac415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ac40d5a80 0x7f8ac415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 -- 172.21.15.62:0/2395190577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac4162050 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac8bc2640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c6e0 tx=0x7f8abc00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc003b70 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc003d10 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0052d0 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac4162330 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:15.998+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac41628f0 con 0x7f8ac415f400 2024-04-26T22:27:16.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.000+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac406a360 con 0x7f8ac415f400 2024-04-26T22:27:16.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.000+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8abc005470 con 0x7f8ac415f400 2024-04-26T22:27:16.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.000+0000 7f8ac0ff9640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 0x7f8aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.000+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8abc08b960 con 0x7f8ac415f400 2024-04-26T22:27:16.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.001+0000 7f8ac3fff640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 0x7f8aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.001+0000 7f8ac3fff640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 0x7f8aa406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0011e0 tx=0x7f8aac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:16.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.005+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8abc059e20 con 0x7f8ac415f400 2024-04-26T22:27:16.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.229+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8ac40d28d0 con 0x7f8ac415f400 2024-04-26T22:27:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.872+0000 7f8ac0ff9640 1 -- 172.21.15.62:0/2395190577 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f8abc059720 con 0x7f8ac415f400 2024-04-26T22:27:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 msgr2=0x7f8aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 0x7f8aa406edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0011e0 tx=0x7f8aac009040 comp rx=0 tx=0).stop 2024-04-26T22:27:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 msgr2=0x7f8ac415e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00c6e0 tx=0x7f8abc00cba0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 shutdown_connections 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8aa406c8f0 0x7f8aa406edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ac415f400 0x7f8ac415e280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ac40d5a80 0x7f8ac415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 --2- 172.21.15.62:0/2395190577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ac40b4b20 0x7f8ac41f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 >> 172.21.15.62:0/2395190577 conn(0x7f8ac409e4f0 msgr2=0x7f8ac40d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 shutdown_connections 2024-04-26T22:27:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.875+0000 7f8aca625640 1 -- 172.21.15.62:0/2395190577 wait complete. 2024-04-26T22:27:16.887 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T22:27:16.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:16.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 Processor -- start 2024-04-26T22:27:16.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 -- start start 2024-04-26T22:27:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f281c0d3820 0x7f281c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0d0210 con 0x7f281c162b40 2024-04-26T22:27:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281c0d04f0 con 0x7f281c0b4ba0 2024-04-26T22:27:16.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.992+0000 7f2822578640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f281c0d41c0 con 0x7f281c0d3820 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f281c162b40 0x7f281c0d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56346/0 (socket says 172.21.15.62:56346) 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 -- 172.21.15.62:0/1585543763 learned_addr learned my addr 172.21.15.62:0/1585543763 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189966002 0 0) 0x7f281c0d0210 con 0x7f281c162b40 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2804003520 con 0x7f281c162b40 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281b7fe640 1 --2- 172.21.15.62:0/1585543763 >> v2:172.21.15.174:3300/0 conn(0x7f281c0d3820 0x7f281c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3322923452 0 0) 0x7f2804003520 con 0x7f281c162b40 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 >> v2:172.21.15.174:3300/0 conn(0x7f281c0d3820 msgr2=0x7f281c0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 --2- 172.21.15.62:0/1585543763 >> v2:172.21.15.174:3300/0 conn(0x7f281c0d3820 0x7f281c0d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 >> v1:172.21.15.174:6789/0 conn(0x7f281c0b4ba0 legacy=0x7f281c0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c0d07d0 con 0x7f281c162b40 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f2810001670 con 0x7f281c162b40 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 >> v1:172.21.15.62:6789/0 conn(0x7f281c162b40 legacy=0x7f281c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 0x7f2804003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.996 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2804004800 0x7f2804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28040051c0 0x7f280400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f280400c0c0 con 0x7f28040039b0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f280400c270 con 0x7f2804004800 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f280400c470 con 0x7f28040051c0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 0x7f2804003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281b7fe640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2804004800 0x7f2804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f2820b15640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28040051c0 0x7f280400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 -- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28040051c0 msgr2=0x7f280400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28040051c0 0x7f280400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 -- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2804004800 msgr2=0x7f2804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2804004800 0x7f2804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.993+0000 7f281bfff640 1 -- 172.21.15.62:0/1585543763 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2804003520 con 0x7f28040039b0 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f281b7fe640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2804004800 0x7f2804004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f2820b15640 1 -- 172.21.15.62:0/1585543763 reap_dead start 2024-04-26T22:27:16.997 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f281bfff640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 0x7f2804003db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2810007fd0 tx=0x7f2804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719f0ebb3ca2aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810007d50 con 0x7f28040039b0 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28100058d0 con 0x7f28040039b0 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.994+0000 7f281affd640 1 -- 172.21.15.62:0/1585543763 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810005d20 con 0x7f28040039b0 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 -- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 msgr2=0x7f2804003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 0x7f2804003db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2810007fd0 tx=0x7f2804005830 comp rx=0 tx=0).stop 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 -- 172.21.15.62:0/1585543763 shutdown_connections 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 --2- 172.21.15.62:0/1585543763 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f28040039b0 0x7f2804003db0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 -- 172.21.15.62:0/1585543763 >> 172.21.15.62:0/1585543763 conn(0x7f281c09e550 msgr2=0x7f281c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 -- 172.21.15.62:0/1585543763 shutdown_connections 2024-04-26T22:27:16.998 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.995+0000 7f2822578640 1 -- 172.21.15.62:0/1585543763 wait complete. 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.996+0000 7f2822578640 1 Processor -- start 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.996+0000 7f2822578640 1 -- start start 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 0x7f281c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281c0ba0b0 0x7f281c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f281c0d12b0 con 0x7f281c0b2e00 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f281c0d0d90 con 0x7f28040039b0 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2822578640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f281c0cfe30 con 0x7f281c0ba0b0 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35306/0 (socket says 172.21.15.62:35306) 2024-04-26T22:27:17.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 -- 172.21.15.62:0/301367305 learned_addr learned my addr 172.21.15.62:0/301367305 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f281b7fe640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281c0ba0b0 0x7f281c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f281bfff640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 0x7f281c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 -- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281c0ba0b0 msgr2=0x7f281c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281c0ba0b0 0x7f281c0c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 -- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 msgr2=0x7f281c150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 0x7f281c150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f2820b15640 1 -- 172.21.15.62:0/301367305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c0b9b90 con 0x7f281c0b2e00 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.997+0000 7f281bfff640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 0x7f281c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2820b15640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f281c09ec30 tx=0x7f281400ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281401b070 con 0x7f281c0b2e00 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2814003df0 con 0x7f281c0b2e00 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c0c9460 con 0x7f281c0b2e00 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814013640 con 0x7f281c0b2e00 2024-04-26T22:27:17.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.998+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c0ca480 con 0x7f281c0b2e00 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.999+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c06a360 con 0x7f281c0b2e00 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:16.999+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f281401e070 con 0x7f281c0b2e00 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.000+0000 7f2818ff9640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 0x7f280806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.000+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f281408b390 con 0x7f281c0b2e00 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.000+0000 7f281bfff640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 0x7f280806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.000+0000 7f281bfff640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 0x7f280806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2810033000 tx=0x7f281002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.004+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f28140597c0 con 0x7f281c0b2e00 2024-04-26T22:27:17.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.225+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f281c0d2960 con 0x7f281c0b2e00 2024-04-26T22:27:17.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.226+0000 7f2818ff9640 1 -- 172.21.15.62:0/301367305 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f28140590c0 con 0x7f281c0b2e00 2024-04-26T22:27:17.226 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:17.226 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:17.226 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 msgr2=0x7f280806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 0x7f280806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2810033000 tx=0x7f281002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 msgr2=0x7f281c0b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f281c09ec30 tx=0x7f281400ea00 comp rx=0 tx=0).stop 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 shutdown_connections 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f280806c920 0x7f280806ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f281c0b2e00 0x7f281c0b3280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f281c0ba0b0 0x7f281c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 --2- 172.21.15.62:0/301367305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28040039b0 0x7f281c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 >> 172.21.15.62:0/301367305 conn(0x7f281c09e550 msgr2=0x7f281c0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:17.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 shutdown_connections 2024-04-26T22:27:17.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.228+0000 7f2822578640 1 -- 172.21.15.62:0/301367305 wait complete. 2024-04-26T22:27:17.239 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:17.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.339+0000 7fc98e187640 1 Processor -- start 2024-04-26T22:27:17.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.339+0000 7fc98e187640 1 -- start start 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc98e187640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9880d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc98e187640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880cfea0 con 0x7fc9881639e0 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc98e187640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9880d0140 con 0x7fc98815f470 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc98e187640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880d0a70 con 0x7fc988163560 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc987fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc9881639e0 0x7fc9880d3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56352/0 (socket says 172.21.15.62:56352) 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc987fff640 1 -- 172.21.15.62:0/2319101725 learned_addr learned my addr 172.21.15.62:0/2319101725 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc986ffd640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9880d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528580346 0 0) 0x7fc9880cfea0 con 0x7fc9881639e0 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc970003520 con 0x7fc9881639e0 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3976852876 0 0) 0x7fc970003520 con 0x7fc9881639e0 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 msgr2=0x7fc9880d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:17.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9880d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 >> v1:172.21.15.174:6789/0 conn(0x7fc98815f470 legacy=0x7fc988162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880d0420 con 0x7fc9881639e0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fc9800019f0 con 0x7fc9881639e0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 >> v1:172.21.15.62:6789/0 conn(0x7fc9881639e0 legacy=0x7fc9880d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9700039e0 0x7fc970003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc970004800 0x7fc970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880d0a70 con 0x7fc9700039e0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc97000ba70 con 0x7fc970004800 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.340+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc97000bc70 con 0x7fc988163560 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9700039e0 0x7fc970003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc986ffd640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc970004800 0x7fc970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc987fff640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 -- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 msgr2=0x7fc9700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:17.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc988163560 0x7fc9700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 -- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc970004800 msgr2=0x7fc970004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc970004800 0x7fc970004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.341+0000 7fc9877fe640 1 -- 172.21.15.62:0/2319101725 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc970003520 con 0x7fc9700039e0 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc9877fe640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9700039e0 0x7fc970003e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc978007fd0 tx=0x7fc970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0f51fd716425f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780013c0 con 0x7fc9700039e0 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9780052d0 con 0x7fc9700039e0 2024-04-26T22:27:17.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc9867fc640 1 -- 172.21.15.62:0/2319101725 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97802a870 con 0x7fc9700039e0 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 -- 172.21.15.62:0/2319101725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9700039e0 msgr2=0x7fc970003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 --2- 172.21.15.62:0/2319101725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9700039e0 0x7fc970003e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc978007fd0 tx=0x7fc970005830 comp rx=0 tx=0).stop 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc987fff640 1 -- 172.21.15.62:0/2319101725 reap_dead start 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 -- 172.21.15.62:0/2319101725 shutdown_connections 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 -- 172.21.15.62:0/2319101725 >> 172.21.15.62:0/2319101725 conn(0x7fc98809e4f0 msgr2=0x7fc9880d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:17.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 -- 172.21.15.62:0/2319101725 shutdown_connections 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.342+0000 7fc98e187640 1 -- 172.21.15.62:0/2319101725 wait complete. 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.343+0000 7fc98e187640 1 Processor -- start 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 -- start start 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9700039e0 0x7fc9881541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9880cdda0 0x7fc9880ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880d5e10 con 0x7fc9880cd930 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9880d5c90 con 0x7fc9880cdda0 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc98e187640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc9880d0f00 con 0x7fc9700039e0 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35324/0 (socket says 172.21.15.62:35324) 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc986ffd640 1 -- 172.21.15.62:0/1835811579 learned_addr learned my addr 172.21.15.62:0/1835811579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc9877fe640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9700039e0 0x7fc9881541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.344+0000 7fc987fff640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9880cdda0 0x7fc9880ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 -- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9700039e0 msgr2=0x7fc9881541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9700039e0 0x7fc9881541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 -- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9880cdda0 msgr2=0x7fc9880ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9880cdda0 0x7fc9880ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 -- 172.21.15.62:0/1835811579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9880ca3e0 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc986ffd640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0119a0 tx=0x7fc97c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c00faf0 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97c00fc90 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9880c9340 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c014440 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.345+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9880c9900 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.346+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc98806a360 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.346+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc97c012040 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.347+0000 7fc96ffff640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.347+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc97c08bed0 con 0x7fc9880cd930 2024-04-26T22:27:17.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.347+0000 7fc9877fe640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.348+0000 7fc9877fe640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 0x7fc97406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc978032000 tx=0x7fc97802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.351+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc97c019030 con 0x7fc9880cd930 2024-04-26T22:27:17.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.536+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9880d2770 con 0x7fc9880cd930 2024-04-26T22:27:17.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.537+0000 7fc96ffff640 1 -- 172.21.15.62:0/1835811579 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fc97c059d40 con 0x7fc9880cd930 2024-04-26T22:27:17.537 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:17.537 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 msgr2=0x7fc97406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 0x7fc97406ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc978032000 tx=0x7fc97802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 msgr2=0x7fc9880caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0119a0 tx=0x7fc97c011e60 comp rx=0 tx=0).stop 2024-04-26T22:27:17.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 shutdown_connections 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9880cdda0 0x7fc9880ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9880cd930 0x7fc9880caf70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 --2- 172.21.15.62:0/1835811579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9700039e0 0x7fc9881541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 >> 172.21.15.62:0/1835811579 conn(0x7fc98809e4f0 msgr2=0x7fc988162e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 shutdown_connections 2024-04-26T22:27:17.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.539+0000 7fc98e187640 1 -- 172.21.15.62:0/1835811579 wait complete. 2024-04-26T22:27:17.550 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:27:17.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:27:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.650+0000 7f694897f640 1 Processor -- start 2024-04-26T22:27:17.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.650+0000 7f694897f640 1 -- start start 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f694897f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440d1c50 0x7f69440c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f694897f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69440ce860 0x7f69440c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f694897f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69440ced60 con 0x7f694415d310 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f694897f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f69440c6ab0 con 0x7f69440ce860 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f694897f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f69440cf040 con 0x7f69440d1c50 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.651+0000 7f6942d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69440ce860 0x7f69440c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f69440ce860 0x7f69440c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35336/0 (socket says 172.21.15.62:35336) 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 -- 172.21.15.62:0/3224141555 learned_addr learned my addr 172.21.15.62:0/3224141555 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6941d9b640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440d1c50 0x7f69440c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 -- 172.21.15.62:0/3224141555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440d1c50 msgr2=0x7f69440c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440d1c50 0x7f69440c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 -- 172.21.15.62:0/3224141555 >> v1:172.21.15.174:6789/0 conn(0x7f694415d310 legacy=0x7f69440d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:17.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 -- 172.21.15.62:0/3224141555 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440cf320 con 0x7f69440ce860 2024-04-26T22:27:17.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.652+0000 7f6942d9d640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69440ce860 0x7f69440c7740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f693c009e50 tx=0x7f693c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc74b6e4b360dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694159a640 1 -- 172.21.15.62:0/3224141555 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c036070 con 0x7f69440ce860 2024-04-26T22:27:17.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694159a640 1 -- 172.21.15.62:0/3224141555 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c031600 con 0x7f69440ce860 2024-04-26T22:27:17.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694159a640 1 -- 172.21.15.62:0/3224141555 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c0319d0 con 0x7f69440ce860 2024-04-26T22:27:17.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 -- 172.21.15.62:0/3224141555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69440ce860 msgr2=0x7f69440c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69440ce860 0x7f69440c7740 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f693c009e50 tx=0x7f693c02a6c0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 -- 172.21.15.62:0/3224141555 shutdown_connections 2024-04-26T22:27:17.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f69440ce860 0x7f69440c7740 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 --2- 172.21.15.62:0/3224141555 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440d1c50 0x7f69440c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 -- 172.21.15.62:0/3224141555 >> 172.21.15.62:0/3224141555 conn(0x7f69440b4a50 msgr2=0x7f69440b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 -- 172.21.15.62:0/3224141555 shutdown_connections 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.653+0000 7f694897f640 1 -- 172.21.15.62:0/3224141555 wait complete. 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 Processor -- start 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 -- start start 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440ce860 0x7f694414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 0x7f69440cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f69440b5970 con 0x7f694415d310 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69440b57f0 con 0x7f69440d1c50 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.655+0000 7f694897f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f694415d6f0 con 0x7f69440ce860 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35352/0 (socket says 172.21.15.62:35352) 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 -- 172.21.15.62:0/3056573318 learned_addr learned my addr 172.21.15.62:0/3056573318 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f694259c640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440ce860 0x7f694414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6941d9b640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 0x7f69440cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 -- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440ce860 msgr2=0x7f694414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440ce860 0x7f694414e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 -- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 msgr2=0x7f69440cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 0x7f69440cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 -- 172.21.15.62:0/3056573318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6944157f20 con 0x7f694415d310 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6941d9b640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 0x7f69440cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6942d9d640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f693c009e50 tx=0x7f693c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c036050 con 0x7f694415d310 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c042070 con 0x7f694415d310 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6944158200 con 0x7f694415d310 2024-04-26T22:27:17.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c009510 con 0x7f694415d310 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.656+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441587c0 con 0x7f694415d310 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.657+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69440693e0 con 0x7f694415d310 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.658+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f693c007bd0 con 0x7f694415d310 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.658+0000 7f6932ffd640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.658+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f693c0ac8c0 con 0x7f694415d310 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.658+0000 7f694259c640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.659+0000 7f694259c640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 0x7f692406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:17.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.662+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f693c07ad80 con 0x7f694415d310 2024-04-26T22:27:17.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.844+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69440c71c0 con 0x7f692406c8f0 2024-04-26T22:27:17.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.851+0000 7f6932ffd640 1 -- 172.21.15.62:0/3056573318 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f69440c71c0 con 0x7f692406c8f0 2024-04-26T22:27:17.854 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:17.859 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 msgr2=0x7f692406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 0x7f692406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6934006fd0 tx=0x7f6934009060 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 msgr2=0x7f69440c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f693c009e50 tx=0x7f693c004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 shutdown_connections 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f692406c8f0 0x7f692406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f694415d310 0x7f69440c8c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69440d1c50 0x7f69440cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 --2- 172.21.15.62:0/3056573318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69440ce860 0x7f694414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 >> 172.21.15.62:0/3056573318 conn(0x7f69440b4a50 msgr2=0x7f69440cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 shutdown_connections 2024-04-26T22:27:17.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:17.854+0000 7f694897f640 1 -- 172.21.15.62:0/3056573318 wait complete. 2024-04-26T22:27:17.865 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T22:27:17.638529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794543"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985465+0000","last_change":"2024-04-26T22:27:13.884599+0000","last_active":"2024-04-26T22:27:14.985465+0000","last_peered":"2024-04-26T22:27:14.985465+0000","last_clean":"2024-04-26T22:27:14.985465+0000","last_became_active":"2024-04-26T22:27:13.884227+0000","last_became_peered":"2024-04-26T22:27:13.884227+0000","last_unstale":"2024-04-26T22:27:14.985465+0000","last_undegraded":"2024-04-26T22:27:14.985465+0000","last_fullsized":"2024-04-26T22:27:14.985465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:12.033642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861849+0000","last_change":"2024-04-26T22:27:13.869959+0000","last_active":"2024-04-26T22:27:14.861849+0000","last_peered":"2024-04-26T22:27:14.861849+0000","last_clean":"2024-04-26T22:27:14.861849+0000","last_became_active":"2024-04-26T22:27:13.869703+0000","last_became_peered":"2024-04-26T22:27:13.869703+0000","last_unstale":"2024-04-26T22:27:14.861849+0000","last_undegraded":"2024-04-26T22:27:14.861849+0000","last_fullsized":"2024-04-26T22:27:14.861849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985260+0000","last_change":"2024-04-26T22:27:13.868365+0000","last_active":"2024-04-26T22:27:14.985260+0000","last_peered":"2024-04-26T22:27:14.985260+0000","last_clean":"2024-04-26T22:27:14.985260+0000","last_became_active":"2024-04-26T22:27:13.868069+0000","last_became_peered":"2024-04-26T22:27:13.868069+0000","last_unstale":"2024-04-26T22:27:14.985260+0000","last_undegraded":"2024-04-26T22:27:14.985260+0000","last_fullsized":"2024-04-26T22:27:14.985260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864150+0000","last_change":"2024-04-26T22:27:13.883158+0000","last_active":"2024-04-26T22:27:14.864150+0000","last_peered":"2024-04-26T22:27:14.864150+0000","last_clean":"2024-04-26T22:27:14.864150+0000","last_became_active":"2024-04-26T22:27:13.882738+0000","last_became_peered":"2024-04-26T22:27:13.882738+0000","last_unstale":"2024-04-26T22:27:14.864150+0000","last_undegraded":"2024-04-26T22:27:14.864150+0000","last_fullsized":"2024-04-26T22:27:14.864150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985094+0000","last_change":"2024-04-26T22:27:13.881003+0000","last_active":"2024-04-26T22:27:14.985094+0000","last_peered":"2024-04-26T22:27:14.985094+0000","last_clean":"2024-04-26T22:27:14.985094+0000","last_became_active":"2024-04-26T22:27:13.879581+0000","last_became_peered":"2024-04-26T22:27:13.879581+0000","last_unstale":"2024-04-26T22:27:14.985094+0000","last_undegraded":"2024-04-26T22:27:14.985094+0000","last_fullsized":"2024-04-26T22:27:14.985094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864095+0000","last_change":"2024-04-26T22:27:13.886150+0000","last_active":"2024-04-26T22:27:14.864095+0000","last_peered":"2024-04-26T22:27:14.864095+0000","last_clean":"2024-04-26T22:27:14.864095+0000","last_became_active":"2024-04-26T22:27:13.885275+0000","last_became_peered":"2024-04-26T22:27:13.885275+0000","last_unstale":"2024-04-26T22:27:14.864095+0000","last_undegraded":"2024-04-26T22:27:14.864095+0000","last_fullsized":"2024-04-26T22:27:14.864095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984951+0000","last_change":"2024-04-26T22:27:13.879538+0000","last_active":"2024-04-26T22:27:14.984951+0000","last_peered":"2024-04-26T22:27:14.984951+0000","last_clean":"2024-04-26T22:27:14.984951+0000","last_became_active":"2024-04-26T22:27:13.879204+0000","last_became_peered":"2024-04-26T22:27:13.879204+0000","last_unstale":"2024-04-26T22:27:14.984951+0000","last_undegraded":"2024-04-26T22:27:14.984951+0000","last_fullsized":"2024-04-26T22:27:14.984951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985834+0000","last_change":"2024-04-26T22:27:13.887610+0000","last_active":"2024-04-26T22:27:14.985834+0000","last_peered":"2024-04-26T22:27:14.985834+0000","last_clean":"2024-04-26T22:27:14.985834+0000","last_became_active":"2024-04-26T22:27:13.887254+0000","last_became_peered":"2024-04-26T22:27:13.887254+0000","last_unstale":"2024-04-26T22:27:14.985834+0000","last_undegraded":"2024-04-26T22:27:14.985834+0000","last_fullsized":"2024-04-26T22:27:14.985834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985485+0000","last_change":"2024-04-26T22:27:13.889269+0000","last_active":"2024-04-26T22:27:14.985485+0000","last_peered":"2024-04-26T22:27:14.985485+0000","last_clean":"2024-04-26T22:27:14.985485+0000","last_became_active":"2024-04-26T22:27:13.888723+0000","last_became_peered":"2024-04-26T22:27:13.888723+0000","last_unstale":"2024-04-26T22:27:14.985485+0000","last_undegraded":"2024-04-26T22:27:14.985485+0000","last_fullsized":"2024-04-26T22:27:14.985485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:10.483729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879899+0000","last_change":"2024-04-26T22:27:13.888770+0000","last_active":"2024-04-26T22:27:14.879899+0000","last_peered":"2024-04-26T22:27:14.879899+0000","last_clean":"2024-04-26T22:27:14.879899+0000","last_became_active":"2024-04-26T22:27:13.888484+0000","last_became_peered":"2024-04-26T22:27:13.888484+0000","last_unstale":"2024-04-26T22:27:14.879899+0000","last_undegraded":"2024-04-26T22:27:14.879899+0000","last_fullsized":"2024-04-26T22:27:14.879899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879762+0000","last_change":"2024-04-26T22:27:13.889012+0000","last_active":"2024-04-26T22:27:14.879762+0000","last_peered":"2024-04-26T22:27:14.879762+0000","last_clean":"2024-04-26T22:27:14.879762+0000","last_became_active":"2024-04-26T22:27:13.888552+0000","last_became_peered":"2024-04-26T22:27:13.888552+0000","last_unstale":"2024-04-26T22:27:14.879762+0000","last_undegraded":"2024-04-26T22:27:14.879762+0000","last_fullsized":"2024-04-26T22:27:14.879762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863018+0000","last_change":"2024-04-26T22:27:13.883636+0000","last_active":"2024-04-26T22:27:14.863018+0000","last_peered":"2024-04-26T22:27:14.863018+0000","last_clean":"2024-04-26T22:27:14.863018+0000","last_became_active":"2024-04-26T22:27:13.883367+0000","last_became_peered":"2024-04-26T22:27:13.883367+0000","last_unstale":"2024-04-26T22:27:14.863018+0000","last_undegraded":"2024-04-26T22:27:14.863018+0000","last_fullsized":"2024-04-26T22:27:14.863018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:22.436979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985852+0000","last_change":"2024-04-26T22:27:13.883693+0000","last_active":"2024-04-26T22:27:14.985852+0000","last_peered":"2024-04-26T22:27:14.985852+0000","last_clean":"2024-04-26T22:27:14.985852+0000","last_became_active":"2024-04-26T22:27:13.883428+0000","last_became_peered":"2024-04-26T22:27:13.883428+0000","last_unstale":"2024-04-26T22:27:14.985852+0000","last_undegraded":"2024-04-26T22:27:14.985852+0000","last_fullsized":"2024-04-26T22:27:14.985852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984950+0000","last_change":"2024-04-26T22:27:13.867843+0000","last_active":"2024-04-26T22:27:14.984950+0000","last_peered":"2024-04-26T22:27:14.984950+0000","last_clean":"2024-04-26T22:27:14.984950+0000","last_became_active":"2024-04-26T22:27:13.867571+0000","last_became_peered":"2024-04-26T22:27:13.867571+0000","last_unstale":"2024-04-26T22:27:14.984950+0000","last_undegraded":"2024-04-26T22:27:14.984950+0000","last_fullsized":"2024-04-26T22:27:14.984950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.442051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861598+0000","last_change":"2024-04-26T22:27:13.870228+0000","last_active":"2024-04-26T22:27:14.861598+0000","last_peered":"2024-04-26T22:27:14.861598+0000","last_clean":"2024-04-26T22:27:14.861598+0000","last_became_active":"2024-04-26T22:27:13.869325+0000","last_became_peered":"2024-04-26T22:27:13.869325+0000","last_unstale":"2024-04-26T22:27:14.861598+0000","last_undegraded":"2024-04-26T22:27:14.861598+0000","last_fullsized":"2024-04-26T22:27:14.861598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862767+0000","last_change":"2024-04-26T22:27:13.886144+0000","last_active":"2024-04-26T22:27:14.862767+0000","last_peered":"2024-04-26T22:27:14.862767+0000","last_clean":"2024-04-26T22:27:14.862767+0000","last_became_active":"2024-04-26T22:27:13.884929+0000","last_became_peered":"2024-04-26T22:27:13.884929+0000","last_unstale":"2024-04-26T22:27:14.862767+0000","last_undegraded":"2024-04-26T22:27:14.862767+0000","last_fullsized":"2024-04-26T22:27:14.862767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863960+0000","last_change":"2024-04-26T22:27:13.880290+0000","last_active":"2024-04-26T22:27:14.863960+0000","last_peered":"2024-04-26T22:27:14.863960+0000","last_clean":"2024-04-26T22:27:14.863960+0000","last_became_active":"2024-04-26T22:27:13.879946+0000","last_became_peered":"2024-04-26T22:27:13.879946+0000","last_unstale":"2024-04-26T22:27:14.863960+0000","last_undegraded":"2024-04-26T22:27:14.863960+0000","last_fullsized":"2024-04-26T22:27:14.863960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:13.885781+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:13.885512+0000","last_became_peered":"2024-04-26T22:27:13.885512+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985377+0000","last_change":"2024-04-26T22:27:13.880900+0000","last_active":"2024-04-26T22:27:14.985377+0000","last_peered":"2024-04-26T22:27:14.985377+0000","last_clean":"2024-04-26T22:27:14.985377+0000","last_became_active":"2024-04-26T22:27:13.879979+0000","last_became_peered":"2024-04-26T22:27:13.879979+0000","last_unstale":"2024-04-26T22:27:14.985377+0000","last_undegraded":"2024-04-26T22:27:14.985377+0000","last_fullsized":"2024-04-26T22:27:14.985377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879817+0000","last_change":"2024-04-26T22:27:13.882410+0000","last_active":"2024-04-26T22:27:14.879817+0000","last_peered":"2024-04-26T22:27:14.879817+0000","last_clean":"2024-04-26T22:27:14.879817+0000","last_became_active":"2024-04-26T22:27:13.882094+0000","last_became_peered":"2024-04-26T22:27:13.882094+0000","last_unstale":"2024-04-26T22:27:14.879817+0000","last_undegraded":"2024-04-26T22:27:14.879817+0000","last_fullsized":"2024-04-26T22:27:14.879817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985068+0000","last_change":"2024-04-26T22:27:13.880347+0000","last_active":"2024-04-26T22:27:14.985068+0000","last_peered":"2024-04-26T22:27:14.985068+0000","last_clean":"2024-04-26T22:27:14.985068+0000","last_became_active":"2024-04-26T22:27:13.879886+0000","last_became_peered":"2024-04-26T22:27:13.879886+0000","last_unstale":"2024-04-26T22:27:14.985068+0000","last_undegraded":"2024-04-26T22:27:14.985068+0000","last_fullsized":"2024-04-26T22:27:14.985068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863180+0000","last_change":"2024-04-26T22:27:13.887537+0000","last_active":"2024-04-26T22:27:14.863180+0000","last_peered":"2024-04-26T22:27:14.863180+0000","last_clean":"2024-04-26T22:27:14.863180+0000","last_became_active":"2024-04-26T22:27:13.884918+0000","last_became_peered":"2024-04-26T22:27:13.884918+0000","last_unstale":"2024-04-26T22:27:14.863180+0000","last_undegraded":"2024-04-26T22:27:14.863180+0000","last_fullsized":"2024-04-26T22:27:14.863180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985665+0000","last_change":"2024-04-26T22:27:13.871027+0000","last_active":"2024-04-26T22:27:14.985665+0000","last_peered":"2024-04-26T22:27:14.985665+0000","last_clean":"2024-04-26T22:27:14.985665+0000","last_became_active":"2024-04-26T22:27:13.870744+0000","last_became_peered":"2024-04-26T22:27:13.870744+0000","last_unstale":"2024-04-26T22:27:14.985665+0000","last_undegraded":"2024-04-26T22:27:14.985665+0000","last_fullsized":"2024-04-26T22:27:14.985665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862976+0000","last_change":"2024-04-26T22:27:13.881627+0000","last_active":"2024-04-26T22:27:14.862976+0000","last_peered":"2024-04-26T22:27:14.862976+0000","last_clean":"2024-04-26T22:27:14.862976+0000","last_became_active":"2024-04-26T22:27:13.881347+0000","last_became_peered":"2024-04-26T22:27:13.881347+0000","last_unstale":"2024-04-26T22:27:14.862976+0000","last_undegraded":"2024-04-26T22:27:14.862976+0000","last_fullsized":"2024-04-26T22:27:14.862976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:13.869549+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:13.869285+0000","last_became_peered":"2024-04-26T22:27:13.869285+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879982+0000","last_change":"2024-04-26T22:27:13.882698+0000","last_active":"2024-04-26T22:27:14.879982+0000","last_peered":"2024-04-26T22:27:14.879982+0000","last_clean":"2024-04-26T22:27:14.879982+0000","last_became_active":"2024-04-26T22:27:13.882439+0000","last_became_peered":"2024-04-26T22:27:13.882439+0000","last_unstale":"2024-04-26T22:27:14.879982+0000","last_undegraded":"2024-04-26T22:27:14.879982+0000","last_fullsized":"2024-04-26T22:27:14.879982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861681+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.861681+0000","last_peered":"2024-04-26T22:27:14.861681+0000","last_clean":"2024-04-26T22:27:14.861681+0000","last_became_active":"2024-04-26T22:27:13.870898+0000","last_became_peered":"2024-04-26T22:27:13.870898+0000","last_unstale":"2024-04-26T22:27:14.861681+0000","last_undegraded":"2024-04-26T22:27:14.861681+0000","last_fullsized":"2024-04-26T22:27:14.861681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986038+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.986038+0000","last_peered":"2024-04-26T22:27:14.986038+0000","last_clean":"2024-04-26T22:27:14.986038+0000","last_became_active":"2024-04-26T22:27:13.870782+0000","last_became_peered":"2024-04-26T22:27:13.870782+0000","last_unstale":"2024-04-26T22:27:14.986038+0000","last_undegraded":"2024-04-26T22:27:14.986038+0000","last_fullsized":"2024-04-26T22:27:14.986038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985309+0000","last_change":"2024-04-26T22:27:13.873858+0000","last_active":"2024-04-26T22:27:14.985309+0000","last_peered":"2024-04-26T22:27:14.985309+0000","last_clean":"2024-04-26T22:27:14.985309+0000","last_became_active":"2024-04-26T22:27:13.873561+0000","last_became_peered":"2024-04-26T22:27:13.873561+0000","last_unstale":"2024-04-26T22:27:14.985309+0000","last_undegraded":"2024-04-26T22:27:14.985309+0000","last_fullsized":"2024-04-26T22:27:14.985309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863353+0000","last_change":"2024-04-26T22:27:13.885617+0000","last_active":"2024-04-26T22:27:14.863353+0000","last_peered":"2024-04-26T22:27:14.863353+0000","last_clean":"2024-04-26T22:27:14.863353+0000","last_became_active":"2024-04-26T22:27:13.884803+0000","last_became_peered":"2024-04-26T22:27:13.884803+0000","last_unstale":"2024-04-26T22:27:14.863353+0000","last_undegraded":"2024-04-26T22:27:14.863353+0000","last_fullsized":"2024-04-26T22:27:14.863353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984684+0000","last_change":"2024-04-26T22:27:13.885893+0000","last_active":"2024-04-26T22:27:14.984684+0000","last_peered":"2024-04-26T22:27:14.984684+0000","last_clean":"2024-04-26T22:27:14.984684+0000","last_became_active":"2024-04-26T22:27:13.885630+0000","last_became_peered":"2024-04-26T22:27:13.885630+0000","last_unstale":"2024-04-26T22:27:14.984684+0000","last_undegraded":"2024-04-26T22:27:14.984684+0000","last_fullsized":"2024-04-26T22:27:14.984684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879630+0000","last_change":"2024-04-26T22:27:13.887046+0000","last_active":"2024-04-26T22:27:14.879630+0000","last_peered":"2024-04-26T22:27:14.879630+0000","last_clean":"2024-04-26T22:27:14.879630+0000","last_became_active":"2024-04-26T22:27:13.886776+0000","last_became_peered":"2024-04-26T22:27:13.886776+0000","last_unstale":"2024-04-26T22:27:14.879630+0000","last_undegraded":"2024-04-26T22:27:14.879630+0000","last_fullsized":"2024-04-26T22:27:14.879630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863864+0000","last_change":"2024-04-26T22:27:13.877833+0000","last_active":"2024-04-26T22:27:14.863864+0000","last_peered":"2024-04-26T22:27:14.863864+0000","last_clean":"2024-04-26T22:27:14.863864+0000","last_became_active":"2024-04-26T22:27:13.877564+0000","last_became_peered":"2024-04-26T22:27:13.877564+0000","last_unstale":"2024-04-26T22:27:14.863864+0000","last_undegraded":"2024-04-26T22:27:14.863864+0000","last_fullsized":"2024-04-26T22:27:14.863864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985909+0000","last_change":"2024-04-26T22:27:12.870169+0000","last_active":"2024-04-26T22:27:14.985909+0000","last_peered":"2024-04-26T22:27:14.985909+0000","last_clean":"2024-04-26T22:27:14.985909+0000","last_became_active":"2024-04-26T22:27:12.866498+0000","last_became_peered":"2024-04-26T22:27:12.866498+0000","last_unstale":"2024-04-26T22:27:14.985909+0000","last_undegraded":"2024-04-26T22:27:14.985909+0000","last_fullsized":"2024-04-26T22:27:14.985909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985270+0000","last_change":"2024-04-26T22:27:13.870477+0000","last_active":"2024-04-26T22:27:14.985270+0000","last_peered":"2024-04-26T22:27:14.985270+0000","last_clean":"2024-04-26T22:27:14.985270+0000","last_became_active":"2024-04-26T22:27:13.870203+0000","last_became_peered":"2024-04-26T22:27:13.870203+0000","last_unstale":"2024-04-26T22:27:14.985270+0000","last_undegraded":"2024-04-26T22:27:14.985270+0000","last_fullsized":"2024-04-26T22:27:14.985270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879610+0000","last_change":"2024-04-26T22:27:12.866037+0000","last_active":"2024-04-26T22:27:14.879610+0000","last_peered":"2024-04-26T22:27:14.879610+0000","last_clean":"2024-04-26T22:27:14.879610+0000","last_became_active":"2024-04-26T22:27:12.865735+0000","last_became_peered":"2024-04-26T22:27:12.865735+0000","last_unstale":"2024-04-26T22:27:14.879610+0000","last_undegraded":"2024-04-26T22:27:14.879610+0000","last_fullsized":"2024-04-26T22:27:14.879610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985764+0000","last_change":"2024-04-26T22:27:12.870843+0000","last_active":"2024-04-26T22:27:14.985764+0000","last_peered":"2024-04-26T22:27:14.985764+0000","last_clean":"2024-04-26T22:27:14.985764+0000","last_became_active":"2024-04-26T22:27:12.870301+0000","last_became_peered":"2024-04-26T22:27:12.870301+0000","last_unstale":"2024-04-26T22:27:14.985764+0000","last_undegraded":"2024-04-26T22:27:14.985764+0000","last_fullsized":"2024-04-26T22:27:14.985764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.836867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985619+0000","last_change":"2024-04-26T22:27:13.871910+0000","last_active":"2024-04-26T22:27:14.985619+0000","last_peered":"2024-04-26T22:27:14.985619+0000","last_clean":"2024-04-26T22:27:14.985619+0000","last_became_active":"2024-04-26T22:27:13.871645+0000","last_became_peered":"2024-04-26T22:27:13.871645+0000","last_unstale":"2024-04-26T22:27:14.985619+0000","last_undegraded":"2024-04-26T22:27:14.985619+0000","last_fullsized":"2024-04-26T22:27:14.985619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863081+0000","last_change":"2024-04-26T22:27:12.860052+0000","last_active":"2024-04-26T22:27:14.863081+0000","last_peered":"2024-04-26T22:27:14.863081+0000","last_clean":"2024-04-26T22:27:14.863081+0000","last_became_active":"2024-04-26T22:27:12.859367+0000","last_became_peered":"2024-04-26T22:27:12.859367+0000","last_unstale":"2024-04-26T22:27:14.863081+0000","last_undegraded":"2024-04-26T22:27:14.863081+0000","last_fullsized":"2024-04-26T22:27:14.863081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861594+0000","last_change":"2024-04-26T22:27:13.868468+0000","last_active":"2024-04-26T22:27:14.861594+0000","last_peered":"2024-04-26T22:27:14.861594+0000","last_clean":"2024-04-26T22:27:14.861594+0000","last_became_active":"2024-04-26T22:27:13.868172+0000","last_became_peered":"2024-04-26T22:27:13.868172+0000","last_unstale":"2024-04-26T22:27:14.861594+0000","last_undegraded":"2024-04-26T22:27:14.861594+0000","last_fullsized":"2024-04-26T22:27:14.861594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879909+0000","last_change":"2024-04-26T22:27:12.865874+0000","last_active":"2024-04-26T22:27:14.879909+0000","last_peered":"2024-04-26T22:27:14.879909+0000","last_clean":"2024-04-26T22:27:14.879909+0000","last_became_active":"2024-04-26T22:27:12.865593+0000","last_became_peered":"2024-04-26T22:27:12.865593+0000","last_unstale":"2024-04-26T22:27:14.879909+0000","last_undegraded":"2024-04-26T22:27:14.879909+0000","last_fullsized":"2024-04-26T22:27:14.879909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863394+0000","last_change":"2024-04-26T22:27:13.881094+0000","last_active":"2024-04-26T22:27:14.863394+0000","last_peered":"2024-04-26T22:27:14.863394+0000","last_clean":"2024-04-26T22:27:14.863394+0000","last_became_active":"2024-04-26T22:27:13.880021+0000","last_became_peered":"2024-04-26T22:27:13.880021+0000","last_unstale":"2024-04-26T22:27:14.863394+0000","last_undegraded":"2024-04-26T22:27:14.863394+0000","last_fullsized":"2024-04-26T22:27:14.863394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863254+0000","last_change":"2024-04-26T22:27:12.859862+0000","last_active":"2024-04-26T22:27:14.863254+0000","last_peered":"2024-04-26T22:27:14.863254+0000","last_clean":"2024-04-26T22:27:14.863254+0000","last_became_active":"2024-04-26T22:27:12.859130+0000","last_became_peered":"2024-04-26T22:27:12.859130+0000","last_unstale":"2024-04-26T22:27:14.863254+0000","last_undegraded":"2024-04-26T22:27:14.863254+0000","last_fullsized":"2024-04-26T22:27:14.863254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985290+0000","last_change":"2024-04-26T22:27:13.876436+0000","last_active":"2024-04-26T22:27:14.985290+0000","last_peered":"2024-04-26T22:27:14.985290+0000","last_clean":"2024-04-26T22:27:14.985290+0000","last_became_active":"2024-04-26T22:27:13.876123+0000","last_became_peered":"2024-04-26T22:27:13.876123+0000","last_unstale":"2024-04-26T22:27:14.985290+0000","last_undegraded":"2024-04-26T22:27:14.985290+0000","last_fullsized":"2024-04-26T22:27:14.985290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861670+0000","last_change":"2024-04-26T22:27:12.864622+0000","last_active":"2024-04-26T22:27:14.861670+0000","last_peered":"2024-04-26T22:27:14.861670+0000","last_clean":"2024-04-26T22:27:14.861670+0000","last_became_active":"2024-04-26T22:27:12.864324+0000","last_became_peered":"2024-04-26T22:27:12.864324+0000","last_unstale":"2024-04-26T22:27:14.861670+0000","last_undegraded":"2024-04-26T22:27:14.861670+0000","last_fullsized":"2024-04-26T22:27:14.861670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863255+0000","last_change":"2024-04-26T22:27:13.874206+0000","last_active":"2024-04-26T22:27:14.863255+0000","last_peered":"2024-04-26T22:27:14.863255+0000","last_clean":"2024-04-26T22:27:14.863255+0000","last_became_active":"2024-04-26T22:27:13.873944+0000","last_became_peered":"2024-04-26T22:27:13.873944+0000","last_unstale":"2024-04-26T22:27:14.863255+0000","last_undegraded":"2024-04-26T22:27:14.863255+0000","last_fullsized":"2024-04-26T22:27:14.863255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:17.274394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879809+0000","last_change":"2024-04-26T22:27:12.865709+0000","last_active":"2024-04-26T22:27:14.879809+0000","last_peered":"2024-04-26T22:27:14.879809+0000","last_clean":"2024-04-26T22:27:14.879809+0000","last_became_active":"2024-04-26T22:27:12.865398+0000","last_became_peered":"2024-04-26T22:27:12.865398+0000","last_unstale":"2024-04-26T22:27:14.879809+0000","last_undegraded":"2024-04-26T22:27:14.879809+0000","last_fullsized":"2024-04-26T22:27:14.879809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985173+0000","last_change":"2024-04-26T22:27:13.886280+0000","last_active":"2024-04-26T22:27:14.985173+0000","last_peered":"2024-04-26T22:27:14.985173+0000","last_clean":"2024-04-26T22:27:14.985173+0000","last_became_active":"2024-04-26T22:27:13.886020+0000","last_became_peered":"2024-04-26T22:27:13.886020+0000","last_unstale":"2024-04-26T22:27:14.985173+0000","last_undegraded":"2024-04-26T22:27:14.985173+0000","last_fullsized":"2024-04-26T22:27:14.985173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985378+0000","last_change":"2024-04-26T22:27:12.864771+0000","last_active":"2024-04-26T22:27:14.985378+0000","last_peered":"2024-04-26T22:27:14.985378+0000","last_clean":"2024-04-26T22:27:14.985378+0000","last_became_active":"2024-04-26T22:27:12.864215+0000","last_became_peered":"2024-04-26T22:27:12.864215+0000","last_unstale":"2024-04-26T22:27:14.985378+0000","last_undegraded":"2024-04-26T22:27:14.985378+0000","last_fullsized":"2024-04-26T22:27:14.985378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:09.919428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984993+0000","last_change":"2024-04-26T22:27:13.877703+0000","last_active":"2024-04-26T22:27:14.984993+0000","last_peered":"2024-04-26T22:27:14.984993+0000","last_clean":"2024-04-26T22:27:14.984993+0000","last_became_active":"2024-04-26T22:27:13.877406+0000","last_became_peered":"2024-04-26T22:27:13.877406+0000","last_unstale":"2024-04-26T22:27:14.984993+0000","last_undegraded":"2024-04-26T22:27:14.984993+0000","last_fullsized":"2024-04-26T22:27:14.984993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.849288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863454+0000","last_change":"2024-04-26T22:27:12.860680+0000","last_active":"2024-04-26T22:27:14.863454+0000","last_peered":"2024-04-26T22:27:14.863454+0000","last_clean":"2024-04-26T22:27:14.863454+0000","last_became_active":"2024-04-26T22:27:12.860342+0000","last_became_peered":"2024-04-26T22:27:12.860342+0000","last_unstale":"2024-04-26T22:27:14.863454+0000","last_undegraded":"2024-04-26T22:27:14.863454+0000","last_fullsized":"2024-04-26T22:27:14.863454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985217+0000","last_change":"2024-04-26T22:27:13.887857+0000","last_active":"2024-04-26T22:27:14.985217+0000","last_peered":"2024-04-26T22:27:14.985217+0000","last_clean":"2024-04-26T22:27:14.985217+0000","last_became_active":"2024-04-26T22:27:13.887578+0000","last_became_peered":"2024-04-26T22:27:13.887578+0000","last_unstale":"2024-04-26T22:27:14.985217+0000","last_undegraded":"2024-04-26T22:27:14.985217+0000","last_fullsized":"2024-04-26T22:27:14.985217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879621+0000","last_change":"2024-04-26T22:27:12.863476+0000","last_active":"2024-04-26T22:27:14.879621+0000","last_peered":"2024-04-26T22:27:14.879621+0000","last_clean":"2024-04-26T22:27:14.879621+0000","last_became_active":"2024-04-26T22:27:12.863172+0000","last_became_peered":"2024-04-26T22:27:12.863172+0000","last_unstale":"2024-04-26T22:27:14.879621+0000","last_undegraded":"2024-04-26T22:27:14.879621+0000","last_fullsized":"2024-04-26T22:27:14.879621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879712+0000","last_change":"2024-04-26T22:27:13.885569+0000","last_active":"2024-04-26T22:27:14.879712+0000","last_peered":"2024-04-26T22:27:14.879712+0000","last_clean":"2024-04-26T22:27:14.879712+0000","last_became_active":"2024-04-26T22:27:13.885255+0000","last_became_peered":"2024-04-26T22:27:13.885255+0000","last_unstale":"2024-04-26T22:27:14.879712+0000","last_undegraded":"2024-04-26T22:27:14.879712+0000","last_fullsized":"2024-04-26T22:27:14.879712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984474+0000","last_change":"2024-04-26T22:27:12.856942+0000","last_active":"2024-04-26T22:27:14.984474+0000","last_peered":"2024-04-26T22:27:14.984474+0000","last_clean":"2024-04-26T22:27:14.984474+0000","last_became_active":"2024-04-26T22:27:12.856199+0000","last_became_peered":"2024-04-26T22:27:12.856199+0000","last_unstale":"2024-04-26T22:27:14.984474+0000","last_undegraded":"2024-04-26T22:27:14.984474+0000","last_fullsized":"2024-04-26T22:27:14.984474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984932+0000","last_change":"2024-04-26T22:27:13.888422+0000","last_active":"2024-04-26T22:27:14.984932+0000","last_peered":"2024-04-26T22:27:14.984932+0000","last_clean":"2024-04-26T22:27:14.984932+0000","last_became_active":"2024-04-26T22:27:13.888063+0000","last_became_peered":"2024-04-26T22:27:13.888063+0000","last_unstale":"2024-04-26T22:27:14.984932+0000","last_undegraded":"2024-04-26T22:27:14.984932+0000","last_fullsized":"2024-04-26T22:27:14.984932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863321+0000","last_change":"2024-04-26T22:27:12.863291+0000","last_active":"2024-04-26T22:27:14.863321+0000","last_peered":"2024-04-26T22:27:14.863321+0000","last_clean":"2024-04-26T22:27:14.863321+0000","last_became_active":"2024-04-26T22:27:12.863016+0000","last_became_peered":"2024-04-26T22:27:12.863016+0000","last_unstale":"2024-04-26T22:27:14.863321+0000","last_undegraded":"2024-04-26T22:27:14.863321+0000","last_fullsized":"2024-04-26T22:27:14.863321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985407+0000","last_change":"2024-04-26T22:27:13.880545+0000","last_active":"2024-04-26T22:27:14.985407+0000","last_peered":"2024-04-26T22:27:14.985407+0000","last_clean":"2024-04-26T22:27:14.985407+0000","last_became_active":"2024-04-26T22:27:13.880053+0000","last_became_peered":"2024-04-26T22:27:13.880053+0000","last_unstale":"2024-04-26T22:27:14.985407+0000","last_undegraded":"2024-04-26T22:27:14.985407+0000","last_fullsized":"2024-04-26T22:27:14.985407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984817+0000","last_change":"2024-04-26T22:27:12.856649+0000","last_active":"2024-04-26T22:27:14.984817+0000","last_peered":"2024-04-26T22:27:14.984817+0000","last_clean":"2024-04-26T22:27:14.984817+0000","last_became_active":"2024-04-26T22:27:12.856005+0000","last_became_peered":"2024-04-26T22:27:12.856005+0000","last_unstale":"2024-04-26T22:27:14.984817+0000","last_undegraded":"2024-04-26T22:27:14.984817+0000","last_fullsized":"2024-04-26T22:27:14.984817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985551+0000","last_change":"2024-04-26T22:27:13.888874+0000","last_active":"2024-04-26T22:27:14.985551+0000","last_peered":"2024-04-26T22:27:14.985551+0000","last_clean":"2024-04-26T22:27:14.985551+0000","last_became_active":"2024-04-26T22:27:13.888500+0000","last_became_peered":"2024-04-26T22:27:13.888500+0000","last_unstale":"2024-04-26T22:27:14.985551+0000","last_undegraded":"2024-04-26T22:27:14.985551+0000","last_fullsized":"2024-04-26T22:27:14.985551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:55.006561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861789+0000","last_change":"2024-04-26T22:27:12.863335+0000","last_active":"2024-04-26T22:27:14.861789+0000","last_peered":"2024-04-26T22:27:14.861789+0000","last_clean":"2024-04-26T22:27:14.861789+0000","last_became_active":"2024-04-26T22:27:12.862860+0000","last_became_peered":"2024-04-26T22:27:12.862860+0000","last_unstale":"2024-04-26T22:27:14.861789+0000","last_undegraded":"2024-04-26T22:27:14.861789+0000","last_fullsized":"2024-04-26T22:27:14.861789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864321+0000","last_change":"2024-04-26T22:27:13.882256+0000","last_active":"2024-04-26T22:27:14.864321+0000","last_peered":"2024-04-26T22:27:14.864321+0000","last_clean":"2024-04-26T22:27:14.864321+0000","last_became_active":"2024-04-26T22:27:13.882007+0000","last_became_peered":"2024-04-26T22:27:13.882007+0000","last_unstale":"2024-04-26T22:27:14.864321+0000","last_undegraded":"2024-04-26T22:27:14.864321+0000","last_fullsized":"2024-04-26T22:27:14.864321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984706+0000","last_change":"2024-04-26T22:27:12.866195+0000","last_active":"2024-04-26T22:27:14.984706+0000","last_peered":"2024-04-26T22:27:14.984706+0000","last_clean":"2024-04-26T22:27:14.984706+0000","last_became_active":"2024-04-26T22:27:12.865877+0000","last_became_peered":"2024-04-26T22:27:12.865877+0000","last_unstale":"2024-04-26T22:27:14.984706+0000","last_undegraded":"2024-04-26T22:27:14.984706+0000","last_fullsized":"2024-04-26T22:27:14.984706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985189+0000","last_change":"2024-04-26T22:27:13.887105+0000","last_active":"2024-04-26T22:27:14.985189+0000","last_peered":"2024-04-26T22:27:14.985189+0000","last_clean":"2024-04-26T22:27:14.985189+0000","last_became_active":"2024-04-26T22:27:13.886847+0000","last_became_peered":"2024-04-26T22:27:13.886847+0000","last_unstale":"2024-04-26T22:27:14.985189+0000","last_undegraded":"2024-04-26T22:27:14.985189+0000","last_fullsized":"2024-04-26T22:27:14.985189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984901+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:27:14.984901+0000","last_peered":"2024-04-26T22:27:14.984901+0000","last_clean":"2024-04-26T22:27:14.984901+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:27:14.984901+0000","last_undegraded":"2024-04-26T22:27:14.984901+0000","last_fullsized":"2024-04-26T22:27:14.984901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863214+0000","last_change":"2024-04-26T22:27:12.855353+0000","last_active":"2024-04-26T22:27:14.863214+0000","last_peered":"2024-04-26T22:27:14.863214+0000","last_clean":"2024-04-26T22:27:14.863214+0000","last_became_active":"2024-04-26T22:27:12.854814+0000","last_became_peered":"2024-04-26T22:27:12.854814+0000","last_unstale":"2024-04-26T22:27:14.863214+0000","last_undegraded":"2024-04-26T22:27:14.863214+0000","last_fullsized":"2024-04-26T22:27:14.863214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985768+0000","last_change":"2024-04-26T22:27:13.888534+0000","last_active":"2024-04-26T22:27:14.985768+0000","last_peered":"2024-04-26T22:27:14.985768+0000","last_clean":"2024-04-26T22:27:14.985768+0000","last_became_active":"2024-04-26T22:27:13.888091+0000","last_became_peered":"2024-04-26T22:27:13.888091+0000","last_unstale":"2024-04-26T22:27:14.985768+0000","last_undegraded":"2024-04-26T22:27:14.985768+0000","last_fullsized":"2024-04-26T22:27:14.985768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984835+0000","last_change":"2024-04-26T22:27:12.869665+0000","last_active":"2024-04-26T22:27:14.984835+0000","last_peered":"2024-04-26T22:27:14.984835+0000","last_clean":"2024-04-26T22:27:14.984835+0000","last_became_active":"2024-04-26T22:27:12.869352+0000","last_became_peered":"2024-04-26T22:27:12.869352+0000","last_unstale":"2024-04-26T22:27:14.984835+0000","last_undegraded":"2024-04-26T22:27:14.984835+0000","last_fullsized":"2024-04-26T22:27:14.984835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985026+0000","last_change":"2024-04-26T22:27:13.869644+0000","last_active":"2024-04-26T22:27:14.985026+0000","last_peered":"2024-04-26T22:27:14.985026+0000","last_clean":"2024-04-26T22:27:14.985026+0000","last_became_active":"2024-04-26T22:27:13.869375+0000","last_became_peered":"2024-04-26T22:27:13.869375+0000","last_unstale":"2024-04-26T22:27:14.985026+0000","last_undegraded":"2024-04-26T22:27:14.985026+0000","last_fullsized":"2024-04-26T22:27:14.985026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:39.482073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984581+0000","last_change":"2024-04-26T22:27:12.859680+0000","last_active":"2024-04-26T22:27:14.984581+0000","last_peered":"2024-04-26T22:27:14.984581+0000","last_clean":"2024-04-26T22:27:14.984581+0000","last_became_active":"2024-04-26T22:27:12.857700+0000","last_became_peered":"2024-04-26T22:27:12.857700+0000","last_unstale":"2024-04-26T22:27:14.984581+0000","last_undegraded":"2024-04-26T22:27:14.984581+0000","last_fullsized":"2024-04-26T22:27:14.984581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861737+0000","last_change":"2024-04-26T22:27:13.870073+0000","last_active":"2024-04-26T22:27:14.861737+0000","last_peered":"2024-04-26T22:27:14.861737+0000","last_clean":"2024-04-26T22:27:14.861737+0000","last_became_active":"2024-04-26T22:27:13.869803+0000","last_became_peered":"2024-04-26T22:27:13.869803+0000","last_unstale":"2024-04-26T22:27:14.861737+0000","last_undegraded":"2024-04-26T22:27:14.861737+0000","last_fullsized":"2024-04-26T22:27:14.861737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863550+0000","last_change":"2024-04-26T22:27:12.859881+0000","last_active":"2024-04-26T22:27:14.863550+0000","last_peered":"2024-04-26T22:27:14.863550+0000","last_clean":"2024-04-26T22:27:14.863550+0000","last_became_active":"2024-04-26T22:27:12.859599+0000","last_became_peered":"2024-04-26T22:27:12.859599+0000","last_unstale":"2024-04-26T22:27:14.863550+0000","last_undegraded":"2024-04-26T22:27:14.863550+0000","last_fullsized":"2024-04-26T22:27:14.863550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863194+0000","last_change":"2024-04-26T22:27:13.873492+0000","last_active":"2024-04-26T22:27:14.863194+0000","last_peered":"2024-04-26T22:27:14.863194+0000","last_clean":"2024-04-26T22:27:14.863194+0000","last_became_active":"2024-04-26T22:27:13.873235+0000","last_became_peered":"2024-04-26T22:27:13.873235+0000","last_unstale":"2024-04-26T22:27:14.863194+0000","last_undegraded":"2024-04-26T22:27:14.863194+0000","last_fullsized":"2024-04-26T22:27:14.863194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:57.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861708+0000","last_change":"2024-04-26T22:27:12.863689+0000","last_active":"2024-04-26T22:27:14.861708+0000","last_peered":"2024-04-26T22:27:14.861708+0000","last_clean":"2024-04-26T22:27:14.861708+0000","last_became_active":"2024-04-26T22:27:12.862689+0000","last_became_peered":"2024-04-26T22:27:12.862689+0000","last_unstale":"2024-04-26T22:27:14.861708+0000","last_undegraded":"2024-04-26T22:27:14.861708+0000","last_fullsized":"2024-04-26T22:27:14.861708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863992+0000","last_change":"2024-04-26T22:27:13.882989+0000","last_active":"2024-04-26T22:27:14.863992+0000","last_peered":"2024-04-26T22:27:14.863992+0000","last_clean":"2024-04-26T22:27:14.863992+0000","last_became_active":"2024-04-26T22:27:13.882731+0000","last_became_peered":"2024-04-26T22:27:13.882731+0000","last_unstale":"2024-04-26T22:27:14.863992+0000","last_undegraded":"2024-04-26T22:27:14.863992+0000","last_fullsized":"2024-04-26T22:27:14.863992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985421+0000","last_change":"2024-04-26T22:27:12.869598+0000","last_active":"2024-04-26T22:27:14.985421+0000","last_peered":"2024-04-26T22:27:14.985421+0000","last_clean":"2024-04-26T22:27:14.985421+0000","last_became_active":"2024-04-26T22:27:12.869301+0000","last_became_peered":"2024-04-26T22:27:12.869301+0000","last_unstale":"2024-04-26T22:27:14.985421+0000","last_undegraded":"2024-04-26T22:27:14.985421+0000","last_fullsized":"2024-04-26T22:27:14.985421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861610+0000","last_change":"2024-04-26T22:27:13.868115+0000","last_active":"2024-04-26T22:27:14.861610+0000","last_peered":"2024-04-26T22:27:14.861610+0000","last_clean":"2024-04-26T22:27:14.861610+0000","last_became_active":"2024-04-26T22:27:13.867854+0000","last_became_peered":"2024-04-26T22:27:13.867854+0000","last_unstale":"2024-04-26T22:27:14.861610+0000","last_undegraded":"2024-04-26T22:27:14.861610+0000","last_fullsized":"2024-04-26T22:27:14.861610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879222+0000","last_change":"2024-04-26T22:27:12.867931+0000","last_active":"2024-04-26T22:27:14.879222+0000","last_peered":"2024-04-26T22:27:14.879222+0000","last_clean":"2024-04-26T22:27:14.879222+0000","last_became_active":"2024-04-26T22:27:12.867588+0000","last_became_peered":"2024-04-26T22:27:12.867588+0000","last_unstale":"2024-04-26T22:27:14.879222+0000","last_undegraded":"2024-04-26T22:27:14.879222+0000","last_fullsized":"2024-04-26T22:27:14.879222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986195+0000","last_change":"2024-04-26T22:27:13.877905+0000","last_active":"2024-04-26T22:27:14.986195+0000","last_peered":"2024-04-26T22:27:14.986195+0000","last_clean":"2024-04-26T22:27:14.986195+0000","last_became_active":"2024-04-26T22:27:13.877626+0000","last_became_peered":"2024-04-26T22:27:13.877626+0000","last_unstale":"2024-04-26T22:27:14.986195+0000","last_undegraded":"2024-04-26T22:27:14.986195+0000","last_fullsized":"2024-04-26T22:27:14.986195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879942+0000","last_change":"2024-04-26T22:27:12.866913+0000","last_active":"2024-04-26T22:27:14.879942+0000","last_peered":"2024-04-26T22:27:14.879942+0000","last_clean":"2024-04-26T22:27:14.879942+0000","last_became_active":"2024-04-26T22:27:12.866428+0000","last_became_peered":"2024-04-26T22:27:12.866428+0000","last_unstale":"2024-04-26T22:27:14.879942+0000","last_undegraded":"2024-04-26T22:27:14.879942+0000","last_fullsized":"2024-04-26T22:27:14.879942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985432+0000","last_change":"2024-04-26T22:27:13.867975+0000","last_active":"2024-04-26T22:27:14.985432+0000","last_peered":"2024-04-26T22:27:14.985432+0000","last_clean":"2024-04-26T22:27:14.985432+0000","last_became_active":"2024-04-26T22:27:13.867657+0000","last_became_peered":"2024-04-26T22:27:13.867657+0000","last_unstale":"2024-04-26T22:27:14.985432+0000","last_undegraded":"2024-04-26T22:27:14.985432+0000","last_fullsized":"2024-04-26T22:27:14.985432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:12.870598+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:12.870147+0000","last_became_peered":"2024-04-26T22:27:12.870147+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863751+0000","last_change":"2024-04-26T22:27:13.869914+0000","last_active":"2024-04-26T22:27:14.863751+0000","last_peered":"2024-04-26T22:27:14.863751+0000","last_clean":"2024-04-26T22:27:14.863751+0000","last_became_active":"2024-04-26T22:27:13.869626+0000","last_became_peered":"2024-04-26T22:27:13.869626+0000","last_unstale":"2024-04-26T22:27:14.863751+0000","last_undegraded":"2024-04-26T22:27:14.863751+0000","last_fullsized":"2024-04-26T22:27:14.863751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861709+0000","last_change":"2024-04-26T22:27:12.865574+0000","last_active":"2024-04-26T22:27:14.861709+0000","last_peered":"2024-04-26T22:27:14.861709+0000","last_clean":"2024-04-26T22:27:14.861709+0000","last_became_active":"2024-04-26T22:27:12.865293+0000","last_became_peered":"2024-04-26T22:27:12.865293+0000","last_unstale":"2024-04-26T22:27:14.861709+0000","last_undegraded":"2024-04-26T22:27:14.861709+0000","last_fullsized":"2024-04-26T22:27:14.861709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985793+0000","last_change":"2024-04-26T22:27:13.873495+0000","last_active":"2024-04-26T22:27:14.985793+0000","last_peered":"2024-04-26T22:27:14.985793+0000","last_clean":"2024-04-26T22:27:14.985793+0000","last_became_active":"2024-04-26T22:27:13.873110+0000","last_became_peered":"2024-04-26T22:27:13.873110+0000","last_unstale":"2024-04-26T22:27:14.985793+0000","last_undegraded":"2024-04-26T22:27:14.985793+0000","last_fullsized":"2024-04-26T22:27:14.985793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861881+0000","last_change":"2024-04-26T22:27:12.860253+0000","last_active":"2024-04-26T22:27:14.861881+0000","last_peered":"2024-04-26T22:27:14.861881+0000","last_clean":"2024-04-26T22:27:14.861881+0000","last_became_active":"2024-04-26T22:27:12.859755+0000","last_became_peered":"2024-04-26T22:27:12.859755+0000","last_unstale":"2024-04-26T22:27:14.861881+0000","last_undegraded":"2024-04-26T22:27:14.861881+0000","last_fullsized":"2024-04-26T22:27:14.861881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986034+0000","last_change":"2024-04-26T22:27:13.875140+0000","last_active":"2024-04-26T22:27:14.986034+0000","last_peered":"2024-04-26T22:27:14.986034+0000","last_clean":"2024-04-26T22:27:14.986034+0000","last_became_active":"2024-04-26T22:27:13.872838+0000","last_became_peered":"2024-04-26T22:27:13.872838+0000","last_unstale":"2024-04-26T22:27:14.986034+0000","last_undegraded":"2024-04-26T22:27:14.986034+0000","last_fullsized":"2024-04-26T22:27:14.986034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985243+0000","last_change":"2024-04-26T22:27:12.867920+0000","last_active":"2024-04-26T22:27:14.985243+0000","last_peered":"2024-04-26T22:27:14.985243+0000","last_clean":"2024-04-26T22:27:14.985243+0000","last_became_active":"2024-04-26T22:27:12.867652+0000","last_became_peered":"2024-04-26T22:27:12.867652+0000","last_unstale":"2024-04-26T22:27:14.985243+0000","last_undegraded":"2024-04-26T22:27:14.985243+0000","last_fullsized":"2024-04-26T22:27:14.985243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:36.571464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879782+0000","last_change":"2024-04-26T22:27:13.885168+0000","last_active":"2024-04-26T22:27:14.879782+0000","last_peered":"2024-04-26T22:27:14.879782+0000","last_clean":"2024-04-26T22:27:14.879782+0000","last_became_active":"2024-04-26T22:27:13.884882+0000","last_became_peered":"2024-04-26T22:27:13.884882+0000","last_unstale":"2024-04-26T22:27:14.879782+0000","last_undegraded":"2024-04-26T22:27:14.879782+0000","last_fullsized":"2024-04-26T22:27:14.879782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863833+0000","last_change":"2024-04-26T22:27:13.878648+0000","last_active":"2024-04-26T22:27:14.863833+0000","last_peered":"2024-04-26T22:27:14.863833+0000","last_clean":"2024-04-26T22:27:14.863833+0000","last_became_active":"2024-04-26T22:27:13.878280+0000","last_became_peered":"2024-04-26T22:27:13.878280+0000","last_unstale":"2024-04-26T22:27:14.863833+0000","last_undegraded":"2024-04-26T22:27:14.863833+0000","last_fullsized":"2024-04-26T22:27:14.863833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985097+0000","last_change":"2024-04-26T22:27:12.865334+0000","last_active":"2024-04-26T22:27:14.985097+0000","last_peered":"2024-04-26T22:27:14.985097+0000","last_clean":"2024-04-26T22:27:14.985097+0000","last_became_active":"2024-04-26T22:27:12.865055+0000","last_became_peered":"2024-04-26T22:27:12.865055+0000","last_unstale":"2024-04-26T22:27:14.985097+0000","last_undegraded":"2024-04-26T22:27:14.985097+0000","last_fullsized":"2024-04-26T22:27:14.985097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864217+0000","last_change":"2024-04-26T22:27:13.879604+0000","last_active":"2024-04-26T22:27:14.864217+0000","last_peered":"2024-04-26T22:27:14.864217+0000","last_clean":"2024-04-26T22:27:14.864217+0000","last_became_active":"2024-04-26T22:27:13.879317+0000","last_became_peered":"2024-04-26T22:27:13.879317+0000","last_unstale":"2024-04-26T22:27:14.864217+0000","last_undegraded":"2024-04-26T22:27:14.864217+0000","last_fullsized":"2024-04-26T22:27:14.864217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985330+0000","last_change":"2024-04-26T22:27:12.870741+0000","last_active":"2024-04-26T22:27:14.985330+0000","last_peered":"2024-04-26T22:27:14.985330+0000","last_clean":"2024-04-26T22:27:14.985330+0000","last_became_active":"2024-04-26T22:27:12.870356+0000","last_became_peered":"2024-04-26T22:27:12.870356+0000","last_unstale":"2024-04-26T22:27:14.985330+0000","last_undegraded":"2024-04-26T22:27:14.985330+0000","last_fullsized":"2024-04-26T22:27:14.985330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.747576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:13.888713+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:13.888393+0000","last_became_peered":"2024-04-26T22:27:13.888393+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:12.870271+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:12.869919+0000","last_became_peered":"2024-04-26T22:27:12.869919+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985725+0000","last_change":"2024-04-26T22:27:13.883151+0000","last_active":"2024-04-26T22:27:14.985725+0000","last_peered":"2024-04-26T22:27:14.985725+0000","last_clean":"2024-04-26T22:27:14.985725+0000","last_became_active":"2024-04-26T22:27:13.882875+0000","last_became_peered":"2024-04-26T22:27:13.882875+0000","last_unstale":"2024-04-26T22:27:14.985725+0000","last_undegraded":"2024-04-26T22:27:14.985725+0000","last_fullsized":"2024-04-26T22:27:14.985725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:12.865998+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:12.865683+0000","last_became_peered":"2024-04-26T22:27:12.865683+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:27:17.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:27:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.005+0000 7f9bf8f97640 1 Processor -- start 2024-04-26T22:27:18.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.005+0000 7f9bf8f97640 1 -- start start 2024-04-26T22:27:18.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf8f97640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9bf40d22e0 0x7f9bf40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf8f97640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9bf415f760 0x7f9bf40b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf8f97640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf40d3de0 con 0x7f9bf415f380 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf8f97640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40d43a0 con 0x7f9bf415f760 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf8f97640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf40d40c0 con 0x7f9bf40d22e0 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.006+0000 7f9bf2d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9bf415f760 0x7f9bf40b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9bf415f760 0x7f9bf40b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35356/0 (socket says 172.21.15.62:35356) 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 -- 172.21.15.62:0/888778990 learned_addr learned my addr 172.21.15.62:0/888778990 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 -- 172.21.15.62:0/888778990 >> v2:172.21.15.174:3300/0 conn(0x7f9bf40d22e0 msgr2=0x7f9bf40b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 --2- 172.21.15.62:0/888778990 >> v2:172.21.15.174:3300/0 conn(0x7f9bf40d22e0 0x7f9bf40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 -- 172.21.15.62:0/888778990 >> v1:172.21.15.174:6789/0 conn(0x7f9bf415f380 legacy=0x7f9bf40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 -- 172.21.15.62:0/888778990 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf40b8280 con 0x7f9bf415f760 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.007+0000 7f9bf2d9d640 1 --2- 172.21.15.62:0/888778990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf415f760 0x7f9bf40b8fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009e50 tx=0x7f9bec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c6d47cd438cfee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf159a640 1 -- 172.21.15.62:0/888778990 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec036070 con 0x7f9bf415f760 2024-04-26T22:27:18.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf159a640 1 -- 172.21.15.62:0/888778990 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec031600 con 0x7f9bf415f760 2024-04-26T22:27:18.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf159a640 1 -- 172.21.15.62:0/888778990 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0319d0 con 0x7f9bf415f760 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 -- 172.21.15.62:0/888778990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf415f760 msgr2=0x7f9bf40b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/888778990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf415f760 0x7f9bf40b8fc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009e50 tx=0x7f9bec02a7a0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 -- 172.21.15.62:0/888778990 shutdown_connections 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/888778990 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf415f760 0x7f9bf40b8fc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/888778990 >> v2:172.21.15.174:3300/0 conn(0x7f9bf40d22e0 0x7f9bf40b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 -- 172.21.15.62:0/888778990 >> 172.21.15.62:0/888778990 conn(0x7f9bf409e560 msgr2=0x7f9bf40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 -- 172.21.15.62:0/888778990 shutdown_connections 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.008+0000 7f9bf8f97640 1 -- 172.21.15.62:0/888778990 wait complete. 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 Processor -- start 2024-04-26T22:27:18.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 -- start start 2024-04-26T22:27:18.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bf415f380 0x7f9bf41616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 0x7f9bf414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40d1280 con 0x7f9bf40d22e0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf40d1100 con 0x7f9bf415f380 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf8f97640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf40d5b50 con 0x7f9bf415f760 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35360/0 (socket says 172.21.15.62:35360) 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.010+0000 7f9bf259c640 1 -- 172.21.15.62:0/2788890459 learned_addr learned my addr 172.21.15.62:0/2788890459 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf2d9d640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 0x7f9bf414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf1d9b640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bf415f380 0x7f9bf41616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 -- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 msgr2=0x7f9bf414d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 0x7f9bf414d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 -- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bf415f380 msgr2=0x7f9bf41616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bf415f380 0x7f9bf41616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 -- 172.21.15.62:0/2788890459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4160630 con 0x7f9bf40d22e0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf2d9d640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 0x7f9bf414d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf259c640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c410 tx=0x7f9be400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4017070 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be40035f0 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf4160910 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4003a20 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.011+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf4160ed0 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.012+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf406a360 con 0x7f9bf40d22e0 2024-04-26T22:27:18.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.012+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9be401a070 con 0x7f9bf40d22e0 2024-04-26T22:27:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.013+0000 7f9beaffd640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 0x7f9bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.013+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9be408af70 con 0x7f9bf40d22e0 2024-04-26T22:27:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.013+0000 7f9bf1d9b640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 0x7f9bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.014+0000 7f9bf1d9b640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 0x7f9bd406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cd530 tx=0x7f9bdc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.017 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.017+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9be4059430 con 0x7f9bf40d22e0 2024-04-26T22:27:18.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.198+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf40cd5a0 con 0x7f9bd406c8f0 2024-04-26T22:27:18.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.204+0000 7f9beaffd640 1 -- 172.21.15.62:0/2788890459 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f9bf40cd5a0 con 0x7f9bd406c8f0 2024-04-26T22:27:18.207 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:18.208 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:27:18.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 msgr2=0x7f9bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 0x7f9bd406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cd530 tx=0x7f9bdc008040 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 msgr2=0x7f9bf4160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c410 tx=0x7f9be400c8d0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 shutdown_connections 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bd406c8f0 0x7f9bd406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bf415f760 0x7f9bf414d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bf415f380 0x7f9bf41616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 --2- 172.21.15.62:0/2788890459 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9bf40d22e0 0x7f9bf4160050 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 >> 172.21.15.62:0/2788890459 conn(0x7f9bf409e560 msgr2=0x7f9bf40bab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 shutdown_connections 2024-04-26T22:27:18.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.207+0000 7f9bf8f97640 1 -- 172.21.15.62:0/2788890459 wait complete. 2024-04-26T22:27:18.218 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-26T22:27:17.638529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794543"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985465+0000","last_change":"2024-04-26T22:27:13.884599+0000","last_active":"2024-04-26T22:27:14.985465+0000","last_peered":"2024-04-26T22:27:14.985465+0000","last_clean":"2024-04-26T22:27:14.985465+0000","last_became_active":"2024-04-26T22:27:13.884227+0000","last_became_peered":"2024-04-26T22:27:13.884227+0000","last_unstale":"2024-04-26T22:27:14.985465+0000","last_undegraded":"2024-04-26T22:27:14.985465+0000","last_fullsized":"2024-04-26T22:27:14.985465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:12.033642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861849+0000","last_change":"2024-04-26T22:27:13.869959+0000","last_active":"2024-04-26T22:27:14.861849+0000","last_peered":"2024-04-26T22:27:14.861849+0000","last_clean":"2024-04-26T22:27:14.861849+0000","last_became_active":"2024-04-26T22:27:13.869703+0000","last_became_peered":"2024-04-26T22:27:13.869703+0000","last_unstale":"2024-04-26T22:27:14.861849+0000","last_undegraded":"2024-04-26T22:27:14.861849+0000","last_fullsized":"2024-04-26T22:27:14.861849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985260+0000","last_change":"2024-04-26T22:27:13.868365+0000","last_active":"2024-04-26T22:27:14.985260+0000","last_peered":"2024-04-26T22:27:14.985260+0000","last_clean":"2024-04-26T22:27:14.985260+0000","last_became_active":"2024-04-26T22:27:13.868069+0000","last_became_peered":"2024-04-26T22:27:13.868069+0000","last_unstale":"2024-04-26T22:27:14.985260+0000","last_undegraded":"2024-04-26T22:27:14.985260+0000","last_fullsized":"2024-04-26T22:27:14.985260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864150+0000","last_change":"2024-04-26T22:27:13.883158+0000","last_active":"2024-04-26T22:27:14.864150+0000","last_peered":"2024-04-26T22:27:14.864150+0000","last_clean":"2024-04-26T22:27:14.864150+0000","last_became_active":"2024-04-26T22:27:13.882738+0000","last_became_peered":"2024-04-26T22:27:13.882738+0000","last_unstale":"2024-04-26T22:27:14.864150+0000","last_undegraded":"2024-04-26T22:27:14.864150+0000","last_fullsized":"2024-04-26T22:27:14.864150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985094+0000","last_change":"2024-04-26T22:27:13.881003+0000","last_active":"2024-04-26T22:27:14.985094+0000","last_peered":"2024-04-26T22:27:14.985094+0000","last_clean":"2024-04-26T22:27:14.985094+0000","last_became_active":"2024-04-26T22:27:13.879581+0000","last_became_peered":"2024-04-26T22:27:13.879581+0000","last_unstale":"2024-04-26T22:27:14.985094+0000","last_undegraded":"2024-04-26T22:27:14.985094+0000","last_fullsized":"2024-04-26T22:27:14.985094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864095+0000","last_change":"2024-04-26T22:27:13.886150+0000","last_active":"2024-04-26T22:27:14.864095+0000","last_peered":"2024-04-26T22:27:14.864095+0000","last_clean":"2024-04-26T22:27:14.864095+0000","last_became_active":"2024-04-26T22:27:13.885275+0000","last_became_peered":"2024-04-26T22:27:13.885275+0000","last_unstale":"2024-04-26T22:27:14.864095+0000","last_undegraded":"2024-04-26T22:27:14.864095+0000","last_fullsized":"2024-04-26T22:27:14.864095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984951+0000","last_change":"2024-04-26T22:27:13.879538+0000","last_active":"2024-04-26T22:27:14.984951+0000","last_peered":"2024-04-26T22:27:14.984951+0000","last_clean":"2024-04-26T22:27:14.984951+0000","last_became_active":"2024-04-26T22:27:13.879204+0000","last_became_peered":"2024-04-26T22:27:13.879204+0000","last_unstale":"2024-04-26T22:27:14.984951+0000","last_undegraded":"2024-04-26T22:27:14.984951+0000","last_fullsized":"2024-04-26T22:27:14.984951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985834+0000","last_change":"2024-04-26T22:27:13.887610+0000","last_active":"2024-04-26T22:27:14.985834+0000","last_peered":"2024-04-26T22:27:14.985834+0000","last_clean":"2024-04-26T22:27:14.985834+0000","last_became_active":"2024-04-26T22:27:13.887254+0000","last_became_peered":"2024-04-26T22:27:13.887254+0000","last_unstale":"2024-04-26T22:27:14.985834+0000","last_undegraded":"2024-04-26T22:27:14.985834+0000","last_fullsized":"2024-04-26T22:27:14.985834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985485+0000","last_change":"2024-04-26T22:27:13.889269+0000","last_active":"2024-04-26T22:27:14.985485+0000","last_peered":"2024-04-26T22:27:14.985485+0000","last_clean":"2024-04-26T22:27:14.985485+0000","last_became_active":"2024-04-26T22:27:13.888723+0000","last_became_peered":"2024-04-26T22:27:13.888723+0000","last_unstale":"2024-04-26T22:27:14.985485+0000","last_undegraded":"2024-04-26T22:27:14.985485+0000","last_fullsized":"2024-04-26T22:27:14.985485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:10.483729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879899+0000","last_change":"2024-04-26T22:27:13.888770+0000","last_active":"2024-04-26T22:27:14.879899+0000","last_peered":"2024-04-26T22:27:14.879899+0000","last_clean":"2024-04-26T22:27:14.879899+0000","last_became_active":"2024-04-26T22:27:13.888484+0000","last_became_peered":"2024-04-26T22:27:13.888484+0000","last_unstale":"2024-04-26T22:27:14.879899+0000","last_undegraded":"2024-04-26T22:27:14.879899+0000","last_fullsized":"2024-04-26T22:27:14.879899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879762+0000","last_change":"2024-04-26T22:27:13.889012+0000","last_active":"2024-04-26T22:27:14.879762+0000","last_peered":"2024-04-26T22:27:14.879762+0000","last_clean":"2024-04-26T22:27:14.879762+0000","last_became_active":"2024-04-26T22:27:13.888552+0000","last_became_peered":"2024-04-26T22:27:13.888552+0000","last_unstale":"2024-04-26T22:27:14.879762+0000","last_undegraded":"2024-04-26T22:27:14.879762+0000","last_fullsized":"2024-04-26T22:27:14.879762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863018+0000","last_change":"2024-04-26T22:27:13.883636+0000","last_active":"2024-04-26T22:27:14.863018+0000","last_peered":"2024-04-26T22:27:14.863018+0000","last_clean":"2024-04-26T22:27:14.863018+0000","last_became_active":"2024-04-26T22:27:13.883367+0000","last_became_peered":"2024-04-26T22:27:13.883367+0000","last_unstale":"2024-04-26T22:27:14.863018+0000","last_undegraded":"2024-04-26T22:27:14.863018+0000","last_fullsized":"2024-04-26T22:27:14.863018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:22.436979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985852+0000","last_change":"2024-04-26T22:27:13.883693+0000","last_active":"2024-04-26T22:27:14.985852+0000","last_peered":"2024-04-26T22:27:14.985852+0000","last_clean":"2024-04-26T22:27:14.985852+0000","last_became_active":"2024-04-26T22:27:13.883428+0000","last_became_peered":"2024-04-26T22:27:13.883428+0000","last_unstale":"2024-04-26T22:27:14.985852+0000","last_undegraded":"2024-04-26T22:27:14.985852+0000","last_fullsized":"2024-04-26T22:27:14.985852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984950+0000","last_change":"2024-04-26T22:27:13.867843+0000","last_active":"2024-04-26T22:27:14.984950+0000","last_peered":"2024-04-26T22:27:14.984950+0000","last_clean":"2024-04-26T22:27:14.984950+0000","last_became_active":"2024-04-26T22:27:13.867571+0000","last_became_peered":"2024-04-26T22:27:13.867571+0000","last_unstale":"2024-04-26T22:27:14.984950+0000","last_undegraded":"2024-04-26T22:27:14.984950+0000","last_fullsized":"2024-04-26T22:27:14.984950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.442051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861598+0000","last_change":"2024-04-26T22:27:13.870228+0000","last_active":"2024-04-26T22:27:14.861598+0000","last_peered":"2024-04-26T22:27:14.861598+0000","last_clean":"2024-04-26T22:27:14.861598+0000","last_became_active":"2024-04-26T22:27:13.869325+0000","last_became_peered":"2024-04-26T22:27:13.869325+0000","last_unstale":"2024-04-26T22:27:14.861598+0000","last_undegraded":"2024-04-26T22:27:14.861598+0000","last_fullsized":"2024-04-26T22:27:14.861598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862767+0000","last_change":"2024-04-26T22:27:13.886144+0000","last_active":"2024-04-26T22:27:14.862767+0000","last_peered":"2024-04-26T22:27:14.862767+0000","last_clean":"2024-04-26T22:27:14.862767+0000","last_became_active":"2024-04-26T22:27:13.884929+0000","last_became_peered":"2024-04-26T22:27:13.884929+0000","last_unstale":"2024-04-26T22:27:14.862767+0000","last_undegraded":"2024-04-26T22:27:14.862767+0000","last_fullsized":"2024-04-26T22:27:14.862767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863960+0000","last_change":"2024-04-26T22:27:13.880290+0000","last_active":"2024-04-26T22:27:14.863960+0000","last_peered":"2024-04-26T22:27:14.863960+0000","last_clean":"2024-04-26T22:27:14.863960+0000","last_became_active":"2024-04-26T22:27:13.879946+0000","last_became_peered":"2024-04-26T22:27:13.879946+0000","last_unstale":"2024-04-26T22:27:14.863960+0000","last_undegraded":"2024-04-26T22:27:14.863960+0000","last_fullsized":"2024-04-26T22:27:14.863960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:13.885781+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:13.885512+0000","last_became_peered":"2024-04-26T22:27:13.885512+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985377+0000","last_change":"2024-04-26T22:27:13.880900+0000","last_active":"2024-04-26T22:27:14.985377+0000","last_peered":"2024-04-26T22:27:14.985377+0000","last_clean":"2024-04-26T22:27:14.985377+0000","last_became_active":"2024-04-26T22:27:13.879979+0000","last_became_peered":"2024-04-26T22:27:13.879979+0000","last_unstale":"2024-04-26T22:27:14.985377+0000","last_undegraded":"2024-04-26T22:27:14.985377+0000","last_fullsized":"2024-04-26T22:27:14.985377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879817+0000","last_change":"2024-04-26T22:27:13.882410+0000","last_active":"2024-04-26T22:27:14.879817+0000","last_peered":"2024-04-26T22:27:14.879817+0000","last_clean":"2024-04-26T22:27:14.879817+0000","last_became_active":"2024-04-26T22:27:13.882094+0000","last_became_peered":"2024-04-26T22:27:13.882094+0000","last_unstale":"2024-04-26T22:27:14.879817+0000","last_undegraded":"2024-04-26T22:27:14.879817+0000","last_fullsized":"2024-04-26T22:27:14.879817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985068+0000","last_change":"2024-04-26T22:27:13.880347+0000","last_active":"2024-04-26T22:27:14.985068+0000","last_peered":"2024-04-26T22:27:14.985068+0000","last_clean":"2024-04-26T22:27:14.985068+0000","last_became_active":"2024-04-26T22:27:13.879886+0000","last_became_peered":"2024-04-26T22:27:13.879886+0000","last_unstale":"2024-04-26T22:27:14.985068+0000","last_undegraded":"2024-04-26T22:27:14.985068+0000","last_fullsized":"2024-04-26T22:27:14.985068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863180+0000","last_change":"2024-04-26T22:27:13.887537+0000","last_active":"2024-04-26T22:27:14.863180+0000","last_peered":"2024-04-26T22:27:14.863180+0000","last_clean":"2024-04-26T22:27:14.863180+0000","last_became_active":"2024-04-26T22:27:13.884918+0000","last_became_peered":"2024-04-26T22:27:13.884918+0000","last_unstale":"2024-04-26T22:27:14.863180+0000","last_undegraded":"2024-04-26T22:27:14.863180+0000","last_fullsized":"2024-04-26T22:27:14.863180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985665+0000","last_change":"2024-04-26T22:27:13.871027+0000","last_active":"2024-04-26T22:27:14.985665+0000","last_peered":"2024-04-26T22:27:14.985665+0000","last_clean":"2024-04-26T22:27:14.985665+0000","last_became_active":"2024-04-26T22:27:13.870744+0000","last_became_peered":"2024-04-26T22:27:13.870744+0000","last_unstale":"2024-04-26T22:27:14.985665+0000","last_undegraded":"2024-04-26T22:27:14.985665+0000","last_fullsized":"2024-04-26T22:27:14.985665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862976+0000","last_change":"2024-04-26T22:27:13.881627+0000","last_active":"2024-04-26T22:27:14.862976+0000","last_peered":"2024-04-26T22:27:14.862976+0000","last_clean":"2024-04-26T22:27:14.862976+0000","last_became_active":"2024-04-26T22:27:13.881347+0000","last_became_peered":"2024-04-26T22:27:13.881347+0000","last_unstale":"2024-04-26T22:27:14.862976+0000","last_undegraded":"2024-04-26T22:27:14.862976+0000","last_fullsized":"2024-04-26T22:27:14.862976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:13.869549+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:13.869285+0000","last_became_peered":"2024-04-26T22:27:13.869285+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879982+0000","last_change":"2024-04-26T22:27:13.882698+0000","last_active":"2024-04-26T22:27:14.879982+0000","last_peered":"2024-04-26T22:27:14.879982+0000","last_clean":"2024-04-26T22:27:14.879982+0000","last_became_active":"2024-04-26T22:27:13.882439+0000","last_became_peered":"2024-04-26T22:27:13.882439+0000","last_unstale":"2024-04-26T22:27:14.879982+0000","last_undegraded":"2024-04-26T22:27:14.879982+0000","last_fullsized":"2024-04-26T22:27:14.879982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861681+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.861681+0000","last_peered":"2024-04-26T22:27:14.861681+0000","last_clean":"2024-04-26T22:27:14.861681+0000","last_became_active":"2024-04-26T22:27:13.870898+0000","last_became_peered":"2024-04-26T22:27:13.870898+0000","last_unstale":"2024-04-26T22:27:14.861681+0000","last_undegraded":"2024-04-26T22:27:14.861681+0000","last_fullsized":"2024-04-26T22:27:14.861681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986038+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.986038+0000","last_peered":"2024-04-26T22:27:14.986038+0000","last_clean":"2024-04-26T22:27:14.986038+0000","last_became_active":"2024-04-26T22:27:13.870782+0000","last_became_peered":"2024-04-26T22:27:13.870782+0000","last_unstale":"2024-04-26T22:27:14.986038+0000","last_undegraded":"2024-04-26T22:27:14.986038+0000","last_fullsized":"2024-04-26T22:27:14.986038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985309+0000","last_change":"2024-04-26T22:27:13.873858+0000","last_active":"2024-04-26T22:27:14.985309+0000","last_peered":"2024-04-26T22:27:14.985309+0000","last_clean":"2024-04-26T22:27:14.985309+0000","last_became_active":"2024-04-26T22:27:13.873561+0000","last_became_peered":"2024-04-26T22:27:13.873561+0000","last_unstale":"2024-04-26T22:27:14.985309+0000","last_undegraded":"2024-04-26T22:27:14.985309+0000","last_fullsized":"2024-04-26T22:27:14.985309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863353+0000","last_change":"2024-04-26T22:27:13.885617+0000","last_active":"2024-04-26T22:27:14.863353+0000","last_peered":"2024-04-26T22:27:14.863353+0000","last_clean":"2024-04-26T22:27:14.863353+0000","last_became_active":"2024-04-26T22:27:13.884803+0000","last_became_peered":"2024-04-26T22:27:13.884803+0000","last_unstale":"2024-04-26T22:27:14.863353+0000","last_undegraded":"2024-04-26T22:27:14.863353+0000","last_fullsized":"2024-04-26T22:27:14.863353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984684+0000","last_change":"2024-04-26T22:27:13.885893+0000","last_active":"2024-04-26T22:27:14.984684+0000","last_peered":"2024-04-26T22:27:14.984684+0000","last_clean":"2024-04-26T22:27:14.984684+0000","last_became_active":"2024-04-26T22:27:13.885630+0000","last_became_peered":"2024-04-26T22:27:13.885630+0000","last_unstale":"2024-04-26T22:27:14.984684+0000","last_undegraded":"2024-04-26T22:27:14.984684+0000","last_fullsized":"2024-04-26T22:27:14.984684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879630+0000","last_change":"2024-04-26T22:27:13.887046+0000","last_active":"2024-04-26T22:27:14.879630+0000","last_peered":"2024-04-26T22:27:14.879630+0000","last_clean":"2024-04-26T22:27:14.879630+0000","last_became_active":"2024-04-26T22:27:13.886776+0000","last_became_peered":"2024-04-26T22:27:13.886776+0000","last_unstale":"2024-04-26T22:27:14.879630+0000","last_undegraded":"2024-04-26T22:27:14.879630+0000","last_fullsized":"2024-04-26T22:27:14.879630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863864+0000","last_change":"2024-04-26T22:27:13.877833+0000","last_active":"2024-04-26T22:27:14.863864+0000","last_peered":"2024-04-26T22:27:14.863864+0000","last_clean":"2024-04-26T22:27:14.863864+0000","last_became_active":"2024-04-26T22:27:13.877564+0000","last_became_peered":"2024-04-26T22:27:13.877564+0000","last_unstale":"2024-04-26T22:27:14.863864+0000","last_undegraded":"2024-04-26T22:27:14.863864+0000","last_fullsized":"2024-04-26T22:27:14.863864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985909+0000","last_change":"2024-04-26T22:27:12.870169+0000","last_active":"2024-04-26T22:27:14.985909+0000","last_peered":"2024-04-26T22:27:14.985909+0000","last_clean":"2024-04-26T22:27:14.985909+0000","last_became_active":"2024-04-26T22:27:12.866498+0000","last_became_peered":"2024-04-26T22:27:12.866498+0000","last_unstale":"2024-04-26T22:27:14.985909+0000","last_undegraded":"2024-04-26T22:27:14.985909+0000","last_fullsized":"2024-04-26T22:27:14.985909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985270+0000","last_change":"2024-04-26T22:27:13.870477+0000","last_active":"2024-04-26T22:27:14.985270+0000","last_peered":"2024-04-26T22:27:14.985270+0000","last_clean":"2024-04-26T22:27:14.985270+0000","last_became_active":"2024-04-26T22:27:13.870203+0000","last_became_peered":"2024-04-26T22:27:13.870203+0000","last_unstale":"2024-04-26T22:27:14.985270+0000","last_undegraded":"2024-04-26T22:27:14.985270+0000","last_fullsized":"2024-04-26T22:27:14.985270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879610+0000","last_change":"2024-04-26T22:27:12.866037+0000","last_active":"2024-04-26T22:27:14.879610+0000","last_peered":"2024-04-26T22:27:14.879610+0000","last_clean":"2024-04-26T22:27:14.879610+0000","last_became_active":"2024-04-26T22:27:12.865735+0000","last_became_peered":"2024-04-26T22:27:12.865735+0000","last_unstale":"2024-04-26T22:27:14.879610+0000","last_undegraded":"2024-04-26T22:27:14.879610+0000","last_fullsized":"2024-04-26T22:27:14.879610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985764+0000","last_change":"2024-04-26T22:27:12.870843+0000","last_active":"2024-04-26T22:27:14.985764+0000","last_peered":"2024-04-26T22:27:14.985764+0000","last_clean":"2024-04-26T22:27:14.985764+0000","last_became_active":"2024-04-26T22:27:12.870301+0000","last_became_peered":"2024-04-26T22:27:12.870301+0000","last_unstale":"2024-04-26T22:27:14.985764+0000","last_undegraded":"2024-04-26T22:27:14.985764+0000","last_fullsized":"2024-04-26T22:27:14.985764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.836867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985619+0000","last_change":"2024-04-26T22:27:13.871910+0000","last_active":"2024-04-26T22:27:14.985619+0000","last_peered":"2024-04-26T22:27:14.985619+0000","last_clean":"2024-04-26T22:27:14.985619+0000","last_became_active":"2024-04-26T22:27:13.871645+0000","last_became_peered":"2024-04-26T22:27:13.871645+0000","last_unstale":"2024-04-26T22:27:14.985619+0000","last_undegraded":"2024-04-26T22:27:14.985619+0000","last_fullsized":"2024-04-26T22:27:14.985619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863081+0000","last_change":"2024-04-26T22:27:12.860052+0000","last_active":"2024-04-26T22:27:14.863081+0000","last_peered":"2024-04-26T22:27:14.863081+0000","last_clean":"2024-04-26T22:27:14.863081+0000","last_became_active":"2024-04-26T22:27:12.859367+0000","last_became_peered":"2024-04-26T22:27:12.859367+0000","last_unstale":"2024-04-26T22:27:14.863081+0000","last_undegraded":"2024-04-26T22:27:14.863081+0000","last_fullsized":"2024-04-26T22:27:14.863081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861594+0000","last_change":"2024-04-26T22:27:13.868468+0000","last_active":"2024-04-26T22:27:14.861594+0000","last_peered":"2024-04-26T22:27:14.861594+0000","last_clean":"2024-04-26T22:27:14.861594+0000","last_became_active":"2024-04-26T22:27:13.868172+0000","last_became_peered":"2024-04-26T22:27:13.868172+0000","last_unstale":"2024-04-26T22:27:14.861594+0000","last_undegraded":"2024-04-26T22:27:14.861594+0000","last_fullsized":"2024-04-26T22:27:14.861594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879909+0000","last_change":"2024-04-26T22:27:12.865874+0000","last_active":"2024-04-26T22:27:14.879909+0000","last_peered":"2024-04-26T22:27:14.879909+0000","last_clean":"2024-04-26T22:27:14.879909+0000","last_became_active":"2024-04-26T22:27:12.865593+0000","last_became_peered":"2024-04-26T22:27:12.865593+0000","last_unstale":"2024-04-26T22:27:14.879909+0000","last_undegraded":"2024-04-26T22:27:14.879909+0000","last_fullsized":"2024-04-26T22:27:14.879909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863394+0000","last_change":"2024-04-26T22:27:13.881094+0000","last_active":"2024-04-26T22:27:14.863394+0000","last_peered":"2024-04-26T22:27:14.863394+0000","last_clean":"2024-04-26T22:27:14.863394+0000","last_became_active":"2024-04-26T22:27:13.880021+0000","last_became_peered":"2024-04-26T22:27:13.880021+0000","last_unstale":"2024-04-26T22:27:14.863394+0000","last_undegraded":"2024-04-26T22:27:14.863394+0000","last_fullsized":"2024-04-26T22:27:14.863394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863254+0000","last_change":"2024-04-26T22:27:12.859862+0000","last_active":"2024-04-26T22:27:14.863254+0000","last_peered":"2024-04-26T22:27:14.863254+0000","last_clean":"2024-04-26T22:27:14.863254+0000","last_became_active":"2024-04-26T22:27:12.859130+0000","last_became_peered":"2024-04-26T22:27:12.859130+0000","last_unstale":"2024-04-26T22:27:14.863254+0000","last_undegraded":"2024-04-26T22:27:14.863254+0000","last_fullsized":"2024-04-26T22:27:14.863254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985290+0000","last_change":"2024-04-26T22:27:13.876436+0000","last_active":"2024-04-26T22:27:14.985290+0000","last_peered":"2024-04-26T22:27:14.985290+0000","last_clean":"2024-04-26T22:27:14.985290+0000","last_became_active":"2024-04-26T22:27:13.876123+0000","last_became_peered":"2024-04-26T22:27:13.876123+0000","last_unstale":"2024-04-26T22:27:14.985290+0000","last_undegraded":"2024-04-26T22:27:14.985290+0000","last_fullsized":"2024-04-26T22:27:14.985290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861670+0000","last_change":"2024-04-26T22:27:12.864622+0000","last_active":"2024-04-26T22:27:14.861670+0000","last_peered":"2024-04-26T22:27:14.861670+0000","last_clean":"2024-04-26T22:27:14.861670+0000","last_became_active":"2024-04-26T22:27:12.864324+0000","last_became_peered":"2024-04-26T22:27:12.864324+0000","last_unstale":"2024-04-26T22:27:14.861670+0000","last_undegraded":"2024-04-26T22:27:14.861670+0000","last_fullsized":"2024-04-26T22:27:14.861670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863255+0000","last_change":"2024-04-26T22:27:13.874206+0000","last_active":"2024-04-26T22:27:14.863255+0000","last_peered":"2024-04-26T22:27:14.863255+0000","last_clean":"2024-04-26T22:27:14.863255+0000","last_became_active":"2024-04-26T22:27:13.873944+0000","last_became_peered":"2024-04-26T22:27:13.873944+0000","last_unstale":"2024-04-26T22:27:14.863255+0000","last_undegraded":"2024-04-26T22:27:14.863255+0000","last_fullsized":"2024-04-26T22:27:14.863255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:17.274394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879809+0000","last_change":"2024-04-26T22:27:12.865709+0000","last_active":"2024-04-26T22:27:14.879809+0000","last_peered":"2024-04-26T22:27:14.879809+0000","last_clean":"2024-04-26T22:27:14.879809+0000","last_became_active":"2024-04-26T22:27:12.865398+0000","last_became_peered":"2024-04-26T22:27:12.865398+0000","last_unstale":"2024-04-26T22:27:14.879809+0000","last_undegraded":"2024-04-26T22:27:14.879809+0000","last_fullsized":"2024-04-26T22:27:14.879809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985173+0000","last_change":"2024-04-26T22:27:13.886280+0000","last_active":"2024-04-26T22:27:14.985173+0000","last_peered":"2024-04-26T22:27:14.985173+0000","last_clean":"2024-04-26T22:27:14.985173+0000","last_became_active":"2024-04-26T22:27:13.886020+0000","last_became_peered":"2024-04-26T22:27:13.886020+0000","last_unstale":"2024-04-26T22:27:14.985173+0000","last_undegraded":"2024-04-26T22:27:14.985173+0000","last_fullsized":"2024-04-26T22:27:14.985173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985378+0000","last_change":"2024-04-26T22:27:12.864771+0000","last_active":"2024-04-26T22:27:14.985378+0000","last_peered":"2024-04-26T22:27:14.985378+0000","last_clean":"2024-04-26T22:27:14.985378+0000","last_became_active":"2024-04-26T22:27:12.864215+0000","last_became_peered":"2024-04-26T22:27:12.864215+0000","last_unstale":"2024-04-26T22:27:14.985378+0000","last_undegraded":"2024-04-26T22:27:14.985378+0000","last_fullsized":"2024-04-26T22:27:14.985378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:09.919428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984993+0000","last_change":"2024-04-26T22:27:13.877703+0000","last_active":"2024-04-26T22:27:14.984993+0000","last_peered":"2024-04-26T22:27:14.984993+0000","last_clean":"2024-04-26T22:27:14.984993+0000","last_became_active":"2024-04-26T22:27:13.877406+0000","last_became_peered":"2024-04-26T22:27:13.877406+0000","last_unstale":"2024-04-26T22:27:14.984993+0000","last_undegraded":"2024-04-26T22:27:14.984993+0000","last_fullsized":"2024-04-26T22:27:14.984993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.849288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863454+0000","last_change":"2024-04-26T22:27:12.860680+0000","last_active":"2024-04-26T22:27:14.863454+0000","last_peered":"2024-04-26T22:27:14.863454+0000","last_clean":"2024-04-26T22:27:14.863454+0000","last_became_active":"2024-04-26T22:27:12.860342+0000","last_became_peered":"2024-04-26T22:27:12.860342+0000","last_unstale":"2024-04-26T22:27:14.863454+0000","last_undegraded":"2024-04-26T22:27:14.863454+0000","last_fullsized":"2024-04-26T22:27:14.863454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985217+0000","last_change":"2024-04-26T22:27:13.887857+0000","last_active":"2024-04-26T22:27:14.985217+0000","last_peered":"2024-04-26T22:27:14.985217+0000","last_clean":"2024-04-26T22:27:14.985217+0000","last_became_active":"2024-04-26T22:27:13.887578+0000","last_became_peered":"2024-04-26T22:27:13.887578+0000","last_unstale":"2024-04-26T22:27:14.985217+0000","last_undegraded":"2024-04-26T22:27:14.985217+0000","last_fullsized":"2024-04-26T22:27:14.985217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879621+0000","last_change":"2024-04-26T22:27:12.863476+0000","last_active":"2024-04-26T22:27:14.879621+0000","last_peered":"2024-04-26T22:27:14.879621+0000","last_clean":"2024-04-26T22:27:14.879621+0000","last_became_active":"2024-04-26T22:27:12.863172+0000","last_became_peered":"2024-04-26T22:27:12.863172+0000","last_unstale":"2024-04-26T22:27:14.879621+0000","last_undegraded":"2024-04-26T22:27:14.879621+0000","last_fullsized":"2024-04-26T22:27:14.879621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879712+0000","last_change":"2024-04-26T22:27:13.885569+0000","last_active":"2024-04-26T22:27:14.879712+0000","last_peered":"2024-04-26T22:27:14.879712+0000","last_clean":"2024-04-26T22:27:14.879712+0000","last_became_active":"2024-04-26T22:27:13.885255+0000","last_became_peered":"2024-04-26T22:27:13.885255+0000","last_unstale":"2024-04-26T22:27:14.879712+0000","last_undegraded":"2024-04-26T22:27:14.879712+0000","last_fullsized":"2024-04-26T22:27:14.879712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984474+0000","last_change":"2024-04-26T22:27:12.856942+0000","last_active":"2024-04-26T22:27:14.984474+0000","last_peered":"2024-04-26T22:27:14.984474+0000","last_clean":"2024-04-26T22:27:14.984474+0000","last_became_active":"2024-04-26T22:27:12.856199+0000","last_became_peered":"2024-04-26T22:27:12.856199+0000","last_unstale":"2024-04-26T22:27:14.984474+0000","last_undegraded":"2024-04-26T22:27:14.984474+0000","last_fullsized":"2024-04-26T22:27:14.984474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984932+0000","last_change":"2024-04-26T22:27:13.888422+0000","last_active":"2024-04-26T22:27:14.984932+0000","last_peered":"2024-04-26T22:27:14.984932+0000","last_clean":"2024-04-26T22:27:14.984932+0000","last_became_active":"2024-04-26T22:27:13.888063+0000","last_became_peered":"2024-04-26T22:27:13.888063+0000","last_unstale":"2024-04-26T22:27:14.984932+0000","last_undegraded":"2024-04-26T22:27:14.984932+0000","last_fullsized":"2024-04-26T22:27:14.984932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863321+0000","last_change":"2024-04-26T22:27:12.863291+0000","last_active":"2024-04-26T22:27:14.863321+0000","last_peered":"2024-04-26T22:27:14.863321+0000","last_clean":"2024-04-26T22:27:14.863321+0000","last_became_active":"2024-04-26T22:27:12.863016+0000","last_became_peered":"2024-04-26T22:27:12.863016+0000","last_unstale":"2024-04-26T22:27:14.863321+0000","last_undegraded":"2024-04-26T22:27:14.863321+0000","last_fullsized":"2024-04-26T22:27:14.863321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985407+0000","last_change":"2024-04-26T22:27:13.880545+0000","last_active":"2024-04-26T22:27:14.985407+0000","last_peered":"2024-04-26T22:27:14.985407+0000","last_clean":"2024-04-26T22:27:14.985407+0000","last_became_active":"2024-04-26T22:27:13.880053+0000","last_became_peered":"2024-04-26T22:27:13.880053+0000","last_unstale":"2024-04-26T22:27:14.985407+0000","last_undegraded":"2024-04-26T22:27:14.985407+0000","last_fullsized":"2024-04-26T22:27:14.985407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984817+0000","last_change":"2024-04-26T22:27:12.856649+0000","last_active":"2024-04-26T22:27:14.984817+0000","last_peered":"2024-04-26T22:27:14.984817+0000","last_clean":"2024-04-26T22:27:14.984817+0000","last_became_active":"2024-04-26T22:27:12.856005+0000","last_became_peered":"2024-04-26T22:27:12.856005+0000","last_unstale":"2024-04-26T22:27:14.984817+0000","last_undegraded":"2024-04-26T22:27:14.984817+0000","last_fullsized":"2024-04-26T22:27:14.984817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985551+0000","last_change":"2024-04-26T22:27:13.888874+0000","last_active":"2024-04-26T22:27:14.985551+0000","last_peered":"2024-04-26T22:27:14.985551+0000","last_clean":"2024-04-26T22:27:14.985551+0000","last_became_active":"2024-04-26T22:27:13.888500+0000","last_became_peered":"2024-04-26T22:27:13.888500+0000","last_unstale":"2024-04-26T22:27:14.985551+0000","last_undegraded":"2024-04-26T22:27:14.985551+0000","last_fullsized":"2024-04-26T22:27:14.985551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:55.006561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861789+0000","last_change":"2024-04-26T22:27:12.863335+0000","last_active":"2024-04-26T22:27:14.861789+0000","last_peered":"2024-04-26T22:27:14.861789+0000","last_clean":"2024-04-26T22:27:14.861789+0000","last_became_active":"2024-04-26T22:27:12.862860+0000","last_became_peered":"2024-04-26T22:27:12.862860+0000","last_unstale":"2024-04-26T22:27:14.861789+0000","last_undegraded":"2024-04-26T22:27:14.861789+0000","last_fullsized":"2024-04-26T22:27:14.861789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864321+0000","last_change":"2024-04-26T22:27:13.882256+0000","last_active":"2024-04-26T22:27:14.864321+0000","last_peered":"2024-04-26T22:27:14.864321+0000","last_clean":"2024-04-26T22:27:14.864321+0000","last_became_active":"2024-04-26T22:27:13.882007+0000","last_became_peered":"2024-04-26T22:27:13.882007+0000","last_unstale":"2024-04-26T22:27:14.864321+0000","last_undegraded":"2024-04-26T22:27:14.864321+0000","last_fullsized":"2024-04-26T22:27:14.864321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984706+0000","last_change":"2024-04-26T22:27:12.866195+0000","last_active":"2024-04-26T22:27:14.984706+0000","last_peered":"2024-04-26T22:27:14.984706+0000","last_clean":"2024-04-26T22:27:14.984706+0000","last_became_active":"2024-04-26T22:27:12.865877+0000","last_became_peered":"2024-04-26T22:27:12.865877+0000","last_unstale":"2024-04-26T22:27:14.984706+0000","last_undegraded":"2024-04-26T22:27:14.984706+0000","last_fullsized":"2024-04-26T22:27:14.984706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985189+0000","last_change":"2024-04-26T22:27:13.887105+0000","last_active":"2024-04-26T22:27:14.985189+0000","last_peered":"2024-04-26T22:27:14.985189+0000","last_clean":"2024-04-26T22:27:14.985189+0000","last_became_active":"2024-04-26T22:27:13.886847+0000","last_became_peered":"2024-04-26T22:27:13.886847+0000","last_unstale":"2024-04-26T22:27:14.985189+0000","last_undegraded":"2024-04-26T22:27:14.985189+0000","last_fullsized":"2024-04-26T22:27:14.985189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984901+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:27:14.984901+0000","last_peered":"2024-04-26T22:27:14.984901+0000","last_clean":"2024-04-26T22:27:14.984901+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:27:14.984901+0000","last_undegraded":"2024-04-26T22:27:14.984901+0000","last_fullsized":"2024-04-26T22:27:14.984901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863214+0000","last_change":"2024-04-26T22:27:12.855353+0000","last_active":"2024-04-26T22:27:14.863214+0000","last_peered":"2024-04-26T22:27:14.863214+0000","last_clean":"2024-04-26T22:27:14.863214+0000","last_became_active":"2024-04-26T22:27:12.854814+0000","last_became_peered":"2024-04-26T22:27:12.854814+0000","last_unstale":"2024-04-26T22:27:14.863214+0000","last_undegraded":"2024-04-26T22:27:14.863214+0000","last_fullsized":"2024-04-26T22:27:14.863214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985768+0000","last_change":"2024-04-26T22:27:13.888534+0000","last_active":"2024-04-26T22:27:14.985768+0000","last_peered":"2024-04-26T22:27:14.985768+0000","last_clean":"2024-04-26T22:27:14.985768+0000","last_became_active":"2024-04-26T22:27:13.888091+0000","last_became_peered":"2024-04-26T22:27:13.888091+0000","last_unstale":"2024-04-26T22:27:14.985768+0000","last_undegraded":"2024-04-26T22:27:14.985768+0000","last_fullsized":"2024-04-26T22:27:14.985768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984835+0000","last_change":"2024-04-26T22:27:12.869665+0000","last_active":"2024-04-26T22:27:14.984835+0000","last_peered":"2024-04-26T22:27:14.984835+0000","last_clean":"2024-04-26T22:27:14.984835+0000","last_became_active":"2024-04-26T22:27:12.869352+0000","last_became_peered":"2024-04-26T22:27:12.869352+0000","last_unstale":"2024-04-26T22:27:14.984835+0000","last_undegraded":"2024-04-26T22:27:14.984835+0000","last_fullsized":"2024-04-26T22:27:14.984835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985026+0000","last_change":"2024-04-26T22:27:13.869644+0000","last_active":"2024-04-26T22:27:14.985026+0000","last_peered":"2024-04-26T22:27:14.985026+0000","last_clean":"2024-04-26T22:27:14.985026+0000","last_became_active":"2024-04-26T22:27:13.869375+0000","last_became_peered":"2024-04-26T22:27:13.869375+0000","last_unstale":"2024-04-26T22:27:14.985026+0000","last_undegraded":"2024-04-26T22:27:14.985026+0000","last_fullsized":"2024-04-26T22:27:14.985026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:39.482073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984581+0000","last_change":"2024-04-26T22:27:12.859680+0000","last_active":"2024-04-26T22:27:14.984581+0000","last_peered":"2024-04-26T22:27:14.984581+0000","last_clean":"2024-04-26T22:27:14.984581+0000","last_became_active":"2024-04-26T22:27:12.857700+0000","last_became_peered":"2024-04-26T22:27:12.857700+0000","last_unstale":"2024-04-26T22:27:14.984581+0000","last_undegraded":"2024-04-26T22:27:14.984581+0000","last_fullsized":"2024-04-26T22:27:14.984581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861737+0000","last_change":"2024-04-26T22:27:13.870073+0000","last_active":"2024-04-26T22:27:14.861737+0000","last_peered":"2024-04-26T22:27:14.861737+0000","last_clean":"2024-04-26T22:27:14.861737+0000","last_became_active":"2024-04-26T22:27:13.869803+0000","last_became_peered":"2024-04-26T22:27:13.869803+0000","last_unstale":"2024-04-26T22:27:14.861737+0000","last_undegraded":"2024-04-26T22:27:14.861737+0000","last_fullsized":"2024-04-26T22:27:14.861737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863550+0000","last_change":"2024-04-26T22:27:12.859881+0000","last_active":"2024-04-26T22:27:14.863550+0000","last_peered":"2024-04-26T22:27:14.863550+0000","last_clean":"2024-04-26T22:27:14.863550+0000","last_became_active":"2024-04-26T22:27:12.859599+0000","last_became_peered":"2024-04-26T22:27:12.859599+0000","last_unstale":"2024-04-26T22:27:14.863550+0000","last_undegraded":"2024-04-26T22:27:14.863550+0000","last_fullsized":"2024-04-26T22:27:14.863550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863194+0000","last_change":"2024-04-26T22:27:13.873492+0000","last_active":"2024-04-26T22:27:14.863194+0000","last_peered":"2024-04-26T22:27:14.863194+0000","last_clean":"2024-04-26T22:27:14.863194+0000","last_became_active":"2024-04-26T22:27:13.873235+0000","last_became_peered":"2024-04-26T22:27:13.873235+0000","last_unstale":"2024-04-26T22:27:14.863194+0000","last_undegraded":"2024-04-26T22:27:14.863194+0000","last_fullsized":"2024-04-26T22:27:14.863194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:57.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861708+0000","last_change":"2024-04-26T22:27:12.863689+0000","last_active":"2024-04-26T22:27:14.861708+0000","last_peered":"2024-04-26T22:27:14.861708+0000","last_clean":"2024-04-26T22:27:14.861708+0000","last_became_active":"2024-04-26T22:27:12.862689+0000","last_became_peered":"2024-04-26T22:27:12.862689+0000","last_unstale":"2024-04-26T22:27:14.861708+0000","last_undegraded":"2024-04-26T22:27:14.861708+0000","last_fullsized":"2024-04-26T22:27:14.861708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863992+0000","last_change":"2024-04-26T22:27:13.882989+0000","last_active":"2024-04-26T22:27:14.863992+0000","last_peered":"2024-04-26T22:27:14.863992+0000","last_clean":"2024-04-26T22:27:14.863992+0000","last_became_active":"2024-04-26T22:27:13.882731+0000","last_became_peered":"2024-04-26T22:27:13.882731+0000","last_unstale":"2024-04-26T22:27:14.863992+0000","last_undegraded":"2024-04-26T22:27:14.863992+0000","last_fullsized":"2024-04-26T22:27:14.863992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985421+0000","last_change":"2024-04-26T22:27:12.869598+0000","last_active":"2024-04-26T22:27:14.985421+0000","last_peered":"2024-04-26T22:27:14.985421+0000","last_clean":"2024-04-26T22:27:14.985421+0000","last_became_active":"2024-04-26T22:27:12.869301+0000","last_became_peered":"2024-04-26T22:27:12.869301+0000","last_unstale":"2024-04-26T22:27:14.985421+0000","last_undegraded":"2024-04-26T22:27:14.985421+0000","last_fullsized":"2024-04-26T22:27:14.985421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861610+0000","last_change":"2024-04-26T22:27:13.868115+0000","last_active":"2024-04-26T22:27:14.861610+0000","last_peered":"2024-04-26T22:27:14.861610+0000","last_clean":"2024-04-26T22:27:14.861610+0000","last_became_active":"2024-04-26T22:27:13.867854+0000","last_became_peered":"2024-04-26T22:27:13.867854+0000","last_unstale":"2024-04-26T22:27:14.861610+0000","last_undegraded":"2024-04-26T22:27:14.861610+0000","last_fullsized":"2024-04-26T22:27:14.861610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879222+0000","last_change":"2024-04-26T22:27:12.867931+0000","last_active":"2024-04-26T22:27:14.879222+0000","last_peered":"2024-04-26T22:27:14.879222+0000","last_clean":"2024-04-26T22:27:14.879222+0000","last_became_active":"2024-04-26T22:27:12.867588+0000","last_became_peered":"2024-04-26T22:27:12.867588+0000","last_unstale":"2024-04-26T22:27:14.879222+0000","last_undegraded":"2024-04-26T22:27:14.879222+0000","last_fullsized":"2024-04-26T22:27:14.879222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986195+0000","last_change":"2024-04-26T22:27:13.877905+0000","last_active":"2024-04-26T22:27:14.986195+0000","last_peered":"2024-04-26T22:27:14.986195+0000","last_clean":"2024-04-26T22:27:14.986195+0000","last_became_active":"2024-04-26T22:27:13.877626+0000","last_became_peered":"2024-04-26T22:27:13.877626+0000","last_unstale":"2024-04-26T22:27:14.986195+0000","last_undegraded":"2024-04-26T22:27:14.986195+0000","last_fullsized":"2024-04-26T22:27:14.986195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879942+0000","last_change":"2024-04-26T22:27:12.866913+0000","last_active":"2024-04-26T22:27:14.879942+0000","last_peered":"2024-04-26T22:27:14.879942+0000","last_clean":"2024-04-26T22:27:14.879942+0000","last_became_active":"2024-04-26T22:27:12.866428+0000","last_became_peered":"2024-04-26T22:27:12.866428+0000","last_unstale":"2024-04-26T22:27:14.879942+0000","last_undegraded":"2024-04-26T22:27:14.879942+0000","last_fullsized":"2024-04-26T22:27:14.879942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985432+0000","last_change":"2024-04-26T22:27:13.867975+0000","last_active":"2024-04-26T22:27:14.985432+0000","last_peered":"2024-04-26T22:27:14.985432+0000","last_clean":"2024-04-26T22:27:14.985432+0000","last_became_active":"2024-04-26T22:27:13.867657+0000","last_became_peered":"2024-04-26T22:27:13.867657+0000","last_unstale":"2024-04-26T22:27:14.985432+0000","last_undegraded":"2024-04-26T22:27:14.985432+0000","last_fullsized":"2024-04-26T22:27:14.985432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:12.870598+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:12.870147+0000","last_became_peered":"2024-04-26T22:27:12.870147+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863751+0000","last_change":"2024-04-26T22:27:13.869914+0000","last_active":"2024-04-26T22:27:14.863751+0000","last_peered":"2024-04-26T22:27:14.863751+0000","last_clean":"2024-04-26T22:27:14.863751+0000","last_became_active":"2024-04-26T22:27:13.869626+0000","last_became_peered":"2024-04-26T22:27:13.869626+0000","last_unstale":"2024-04-26T22:27:14.863751+0000","last_undegraded":"2024-04-26T22:27:14.863751+0000","last_fullsized":"2024-04-26T22:27:14.863751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861709+0000","last_change":"2024-04-26T22:27:12.865574+0000","last_active":"2024-04-26T22:27:14.861709+0000","last_peered":"2024-04-26T22:27:14.861709+0000","last_clean":"2024-04-26T22:27:14.861709+0000","last_became_active":"2024-04-26T22:27:12.865293+0000","last_became_peered":"2024-04-26T22:27:12.865293+0000","last_unstale":"2024-04-26T22:27:14.861709+0000","last_undegraded":"2024-04-26T22:27:14.861709+0000","last_fullsized":"2024-04-26T22:27:14.861709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985793+0000","last_change":"2024-04-26T22:27:13.873495+0000","last_active":"2024-04-26T22:27:14.985793+0000","last_peered":"2024-04-26T22:27:14.985793+0000","last_clean":"2024-04-26T22:27:14.985793+0000","last_became_active":"2024-04-26T22:27:13.873110+0000","last_became_peered":"2024-04-26T22:27:13.873110+0000","last_unstale":"2024-04-26T22:27:14.985793+0000","last_undegraded":"2024-04-26T22:27:14.985793+0000","last_fullsized":"2024-04-26T22:27:14.985793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861881+0000","last_change":"2024-04-26T22:27:12.860253+0000","last_active":"2024-04-26T22:27:14.861881+0000","last_peered":"2024-04-26T22:27:14.861881+0000","last_clean":"2024-04-26T22:27:14.861881+0000","last_became_active":"2024-04-26T22:27:12.859755+0000","last_became_peered":"2024-04-26T22:27:12.859755+0000","last_unstale":"2024-04-26T22:27:14.861881+0000","last_undegraded":"2024-04-26T22:27:14.861881+0000","last_fullsized":"2024-04-26T22:27:14.861881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986034+0000","last_change":"2024-04-26T22:27:13.875140+0000","last_active":"2024-04-26T22:27:14.986034+0000","last_peered":"2024-04-26T22:27:14.986034+0000","last_clean":"2024-04-26T22:27:14.986034+0000","last_became_active":"2024-04-26T22:27:13.872838+0000","last_became_peered":"2024-04-26T22:27:13.872838+0000","last_unstale":"2024-04-26T22:27:14.986034+0000","last_undegraded":"2024-04-26T22:27:14.986034+0000","last_fullsized":"2024-04-26T22:27:14.986034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985243+0000","last_change":"2024-04-26T22:27:12.867920+0000","last_active":"2024-04-26T22:27:14.985243+0000","last_peered":"2024-04-26T22:27:14.985243+0000","last_clean":"2024-04-26T22:27:14.985243+0000","last_became_active":"2024-04-26T22:27:12.867652+0000","last_became_peered":"2024-04-26T22:27:12.867652+0000","last_unstale":"2024-04-26T22:27:14.985243+0000","last_undegraded":"2024-04-26T22:27:14.985243+0000","last_fullsized":"2024-04-26T22:27:14.985243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:36.571464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879782+0000","last_change":"2024-04-26T22:27:13.885168+0000","last_active":"2024-04-26T22:27:14.879782+0000","last_peered":"2024-04-26T22:27:14.879782+0000","last_clean":"2024-04-26T22:27:14.879782+0000","last_became_active":"2024-04-26T22:27:13.884882+0000","last_became_peered":"2024-04-26T22:27:13.884882+0000","last_unstale":"2024-04-26T22:27:14.879782+0000","last_undegraded":"2024-04-26T22:27:14.879782+0000","last_fullsized":"2024-04-26T22:27:14.879782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863833+0000","last_change":"2024-04-26T22:27:13.878648+0000","last_active":"2024-04-26T22:27:14.863833+0000","last_peered":"2024-04-26T22:27:14.863833+0000","last_clean":"2024-04-26T22:27:14.863833+0000","last_became_active":"2024-04-26T22:27:13.878280+0000","last_became_peered":"2024-04-26T22:27:13.878280+0000","last_unstale":"2024-04-26T22:27:14.863833+0000","last_undegraded":"2024-04-26T22:27:14.863833+0000","last_fullsized":"2024-04-26T22:27:14.863833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985097+0000","last_change":"2024-04-26T22:27:12.865334+0000","last_active":"2024-04-26T22:27:14.985097+0000","last_peered":"2024-04-26T22:27:14.985097+0000","last_clean":"2024-04-26T22:27:14.985097+0000","last_became_active":"2024-04-26T22:27:12.865055+0000","last_became_peered":"2024-04-26T22:27:12.865055+0000","last_unstale":"2024-04-26T22:27:14.985097+0000","last_undegraded":"2024-04-26T22:27:14.985097+0000","last_fullsized":"2024-04-26T22:27:14.985097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864217+0000","last_change":"2024-04-26T22:27:13.879604+0000","last_active":"2024-04-26T22:27:14.864217+0000","last_peered":"2024-04-26T22:27:14.864217+0000","last_clean":"2024-04-26T22:27:14.864217+0000","last_became_active":"2024-04-26T22:27:13.879317+0000","last_became_peered":"2024-04-26T22:27:13.879317+0000","last_unstale":"2024-04-26T22:27:14.864217+0000","last_undegraded":"2024-04-26T22:27:14.864217+0000","last_fullsized":"2024-04-26T22:27:14.864217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985330+0000","last_change":"2024-04-26T22:27:12.870741+0000","last_active":"2024-04-26T22:27:14.985330+0000","last_peered":"2024-04-26T22:27:14.985330+0000","last_clean":"2024-04-26T22:27:14.985330+0000","last_became_active":"2024-04-26T22:27:12.870356+0000","last_became_peered":"2024-04-26T22:27:12.870356+0000","last_unstale":"2024-04-26T22:27:14.985330+0000","last_undegraded":"2024-04-26T22:27:14.985330+0000","last_fullsized":"2024-04-26T22:27:14.985330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.747576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:13.888713+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:13.888393+0000","last_became_peered":"2024-04-26T22:27:13.888393+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:12.870271+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:12.869919+0000","last_became_peered":"2024-04-26T22:27:12.869919+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985725+0000","last_change":"2024-04-26T22:27:13.883151+0000","last_active":"2024-04-26T22:27:14.985725+0000","last_peered":"2024-04-26T22:27:14.985725+0000","last_clean":"2024-04-26T22:27:14.985725+0000","last_became_active":"2024-04-26T22:27:13.882875+0000","last_became_peered":"2024-04-26T22:27:13.882875+0000","last_unstale":"2024-04-26T22:27:14.985725+0000","last_undegraded":"2024-04-26T22:27:14.985725+0000","last_fullsized":"2024-04-26T22:27:14.985725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:12.865998+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:12.865683+0000","last_became_peered":"2024-04-26T22:27:12.865683+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:27:18.222 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:27:18.222 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T22:27:18.222 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T22:27:18.222 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T22:27:18.223 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T22:27:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.358+0000 7f8507de2640 1 Processor -- start 2024-04-26T22:27:18.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.358+0000 7f8507de2640 1 -- start start 2024-04-26T22:27:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f8507de2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f850015f4f0 0x7f850006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f8507de2640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85000b1480 0x7f85000ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f8507de2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85000b59b0 con 0x7f850006c240 2024-04-26T22:27:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f8507de2640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8500070350 con 0x7f85000b1480 2024-04-26T22:27:18.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f8507de2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f85000b5c70 con 0x7f850015f4f0 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85000b1480 0x7f85000ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f85000b1480 0x7f85000ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35366/0 (socket says 172.21.15.62:35366) 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 -- 172.21.15.62:0/2798742421 learned_addr learned my addr 172.21.15.62:0/2798742421 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 -- 172.21.15.62:0/2798742421 >> v2:172.21.15.174:3300/0 conn(0x7f850015f4f0 msgr2=0x7f850006fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850537d640 1 --2- 172.21.15.62:0/2798742421 >> v2:172.21.15.174:3300/0 conn(0x7f850015f4f0 0x7f850006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 --2- 172.21.15.62:0/2798742421 >> v2:172.21.15.174:3300/0 conn(0x7f850015f4f0 0x7f850006fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 -- 172.21.15.62:0/2798742421 >> v1:172.21.15.174:6789/0 conn(0x7f850006c240 legacy=0x7f850006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:18.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.359+0000 7f850637f640 1 -- 172.21.15.62:0/2798742421 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85000b5f50 con 0x7f85000b1480 2024-04-26T22:27:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.360+0000 7f850637f640 1 --2- 172.21.15.62:0/2798742421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85000b1480 0x7f85000ae8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85000b5900 tx=0x7f84f802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3a1ac61081db72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.360+0000 7f8504b7c640 1 -- 172.21.15.62:0/2798742421 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8036070 con 0x7f85000b1480 2024-04-26T22:27:18.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.360+0000 7f8504b7c640 1 -- 172.21.15.62:0/2798742421 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8031600 con 0x7f85000b1480 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.360+0000 7f8504b7c640 1 -- 172.21.15.62:0/2798742421 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80319d0 con 0x7f85000b1480 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.360+0000 7f8507de2640 1 -- 172.21.15.62:0/2798742421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85000b1480 msgr2=0x7f85000ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 --2- 172.21.15.62:0/2798742421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85000b1480 0x7f85000ae8c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85000b5900 tx=0x7f84f802a6f0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 -- 172.21.15.62:0/2798742421 shutdown_connections 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 --2- 172.21.15.62:0/2798742421 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f85000b1480 0x7f85000ae8c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 --2- 172.21.15.62:0/2798742421 >> v2:172.21.15.174:3300/0 conn(0x7f850015f4f0 0x7f850006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 -- 172.21.15.62:0/2798742421 >> 172.21.15.62:0/2798742421 conn(0x7f85000b3b70 msgr2=0x7f85000b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 -- 172.21.15.62:0/2798742421 shutdown_connections 2024-04-26T22:27:18.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.361+0000 7f8507de2640 1 -- 172.21.15.62:0/2798742421 wait complete. 2024-04-26T22:27:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.362+0000 7f8507de2640 1 Processor -- start 2024-04-26T22:27:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.362+0000 7f8507de2640 1 -- start start 2024-04-26T22:27:18.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.362+0000 7f8507de2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.362+0000 7f8507de2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85000b1480 0x7f85000ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.362+0000 7f8507de2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f850015f4f0 0x7f850014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8507de2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f85000b0fc0 con 0x7f850006c240 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8507de2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85000aee00 con 0x7f85000b1480 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8507de2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f85000b6750 con 0x7f850015f4f0 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35368/0 (socket says 172.21.15.62:35368) 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 -- 172.21.15.62:0/915971569 learned_addr learned my addr 172.21.15.62:0/915971569 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f850637f640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f850015f4f0 0x7f850014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f850537d640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85000b1480 0x7f85000ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 -- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f850015f4f0 msgr2=0x7f850014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f850015f4f0 0x7f850014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 -- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85000b1480 msgr2=0x7f85000ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85000b1480 0x7f85000ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 -- 172.21.15.62:0/915971569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850014ea40 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8505b7e640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c460 tx=0x7f84fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc016020 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc012870 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85000cf290 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc012c60 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.363+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85000cf850 con 0x7f850006c240 2024-04-26T22:27:18.366 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.364+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850006a360 con 0x7f850006c240 2024-04-26T22:27:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.365+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84fc019020 con 0x7f850006c240 2024-04-26T22:27:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.365+0000 7f84f67fc640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 0x7f84e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.365+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f84fc08b660 con 0x7f850006c240 2024-04-26T22:27:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.366+0000 7f850537d640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 0x7f84e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.367 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.366+0000 7f850537d640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 0x7f84e406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8500159220 tx=0x7f84f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.370 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.370+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84fc020070 con 0x7f850006c240 2024-04-26T22:27:18.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.605+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8500073e50 con 0x7f850006c240 2024-04-26T22:27:18.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.608+0000 7f84f67fc640 1 -- 172.21.15.62:0/915971569 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174142 (secure 0 0 0) 0x7f84fc059a00 con 0x7f850006c240 2024-04-26T22:27:18.610 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 msgr2=0x7f84e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 0x7f84e406edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8500159220 tx=0x7f84f0008040 comp rx=0 tx=0).stop 2024-04-26T22:27:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 msgr2=0x7f850015c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c460 tx=0x7f84fc00c920 comp rx=0 tx=0).stop 2024-04-26T22:27:18.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 shutdown_connections 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84e406c8f0 0x7f84e406edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f850015f4f0 0x7f850014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85000b1480 0x7f85000ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 --2- 172.21.15.62:0/915971569 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f850006c240 0x7f850015c4d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 >> 172.21.15.62:0/915971569 conn(0x7f85000b3b70 msgr2=0x7f850015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 shutdown_connections 2024-04-26T22:27:18.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.610+0000 7f8507de2640 1 -- 172.21.15.62:0/915971569 wait complete. 2024-04-26T22:27:18.621 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":1193748239},{"type":"v1","addr":"172.21.15.174:6833","nonce":1193748239}]},"active_addr":"172.21.15.174:6833/1193748239","active_change":"2024-04-26T22:26:53.627594+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2758015735}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":1292623297}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":2631425851}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":83851150}]}]} 2024-04-26T22:27:18.624 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T22:27:18.624 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T22:27:18.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.725+0000 7f26b4a52640 1 Processor -- start 2024-04-26T22:27:18.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.725+0000 7f26b4a52640 1 -- start start 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26b4a52640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f26b0162b40 0x7f26b00d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26b4a52640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26b4a52640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b00d0210 con 0x7f26b00d1850 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26b4a52640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f26b00d41c0 con 0x7f26b0162b40 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26b4a52640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f26b00d04f0 con 0x7f26b00d3820 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26aed9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f26b00d1850 0x7f26b00d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56362/0 (socket says 172.21.15.62:56362) 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26aed9d640 1 -- 172.21.15.62:0/2475005684 learned_addr learned my addr 172.21.15.62:0/2475005684 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:18.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26ae59c640 1 --2- 172.21.15.62:0/2475005684 >> v2:172.21.15.174:3300/0 conn(0x7f26b0162b40 0x7f26b00d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26add9b640 1 --2- 172.21.15.62:0/2475005684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.726+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2041191597 0 0) 0x7f26b00d0210 con 0x7f26b00d1850 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2698003520 con 0x7f26b00d1850 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26add9b640 1 -- 172.21.15.62:0/2475005684 >> v2:172.21.15.174:3300/0 conn(0x7f26b0162b40 msgr2=0x7f26b00d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1442513804 0 0) 0x7f2698003520 con 0x7f26b00d1850 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26add9b640 1 --2- 172.21.15.62:0/2475005684 >> v2:172.21.15.174:3300/0 conn(0x7f26b0162b40 0x7f26b00d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26add9b640 1 -- 172.21.15.62:0/2475005684 >> v1:172.21.15.62:6789/0 conn(0x7f26b00d1850 legacy=0x7f26b00d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.727+0000 7f26add9b640 1 -- 172.21.15.62:0/2475005684 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b00d07d0 con 0x7f26b00d3820 2024-04-26T22:27:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26add9b640 1 --2- 172.21.15.62:0/2475005684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b00d3c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f26a40097a0 tx=0x7f26a402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8502f24ef55081 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4037070 con 0x7f26b00d3820 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a402ac30 con 0x7f26b00d3820 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26ad59a640 1 -- 172.21.15.62:0/2475005684 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00019f0 con 0x7f26b00d3820 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 -- 172.21.15.62:0/2475005684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 msgr2=0x7f26b00d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 --2- 172.21.15.62:0/2475005684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b00d3c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f26a40097a0 tx=0x7f26a402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 -- 172.21.15.62:0/2475005684 shutdown_connections 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 --2- 172.21.15.62:0/2475005684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b00d3c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 --2- 172.21.15.62:0/2475005684 >> v2:172.21.15.174:3300/0 conn(0x7f26b0162b40 0x7f26b00d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 -- 172.21.15.62:0/2475005684 >> 172.21.15.62:0/2475005684 conn(0x7f26b009e550 msgr2=0x7f26b00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 -- 172.21.15.62:0/2475005684 shutdown_connections 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.728+0000 7f26b4a52640 1 -- 172.21.15.62:0/2475005684 wait complete. 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 Processor -- start 2024-04-26T22:27:18.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 -- start start 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b01546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b0162b40 0x7f26b00b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f26b00b5bb0 con 0x7f26b00d1850 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26b00d0d90 con 0x7f26b0162b40 2024-04-26T22:27:18.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26b4a52640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f26b00d6110 con 0x7f26b00d3820 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.730+0000 7f26ae59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35374/0 (socket says 172.21.15.62:35374) 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 -- 172.21.15.62:0/3778984238 learned_addr learned my addr 172.21.15.62:0/3778984238 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 -- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 msgr2=0x7f26b01546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b01546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 -- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b0162b40 msgr2=0x7f26b00b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b0162b40 0x7f26b00b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 -- 172.21.15.62:0/3778984238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b00cd070 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26ae59c640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9b0 tx=0x7f26a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8017070 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a800c9a0 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b00cd350 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a800cd90 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.731+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b00cd910 con 0x7f26b00d1850 2024-04-26T22:27:18.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.732+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b006a360 con 0x7f26b00d1850 2024-04-26T22:27:18.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.733+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26a801a070 con 0x7f26b00d1850 2024-04-26T22:27:18.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.733+0000 7f269effd640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 0x7f269006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:18.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.733+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f26a808bfe0 con 0x7f26b00d1850 2024-04-26T22:27:18.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.733+0000 7f26add9b640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 0x7f269006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:18.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.734+0000 7f26add9b640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 0x7f269006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26a4034000 tx=0x7f26a402c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:18.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.738+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f26a805a390 con 0x7f26b00d1850 2024-04-26T22:27:18.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.926+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26b00d2960 con 0x7f26b00d1850 2024-04-26T22:27:18.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.927+0000 7f269effd640 1 -- 172.21.15.62:0/3778984238 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f26a8059c90 con 0x7f26b00d1850 2024-04-26T22:27:18.927 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:18.927 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:18.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 msgr2=0x7f269006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 0x7f269006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f26a4034000 tx=0x7f26a402c000 comp rx=0 tx=0).stop 2024-04-26T22:27:18.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 msgr2=0x7f26b0151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:18.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f26a800e9b0 tx=0x7f26a800ee70 comp rx=0 tx=0).stop 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 shutdown_connections 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f269006c8f0 0x7f269006edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b0162b40 0x7f26b00b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f26b00d3820 0x7f26b01546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 --2- 172.21.15.62:0/3778984238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f26b00d1850 0x7f26b0151830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 >> 172.21.15.62:0/3778984238 conn(0x7f26b009e550 msgr2=0x7f26b00b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 shutdown_connections 2024-04-26T22:27:18.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:18.929+0000 7f26b4a52640 1 -- 172.21.15.62:0/3778984238 wait complete. 2024-04-26T22:27:18.940 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T22:27:18.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:19.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.040+0000 7f06b7c7e640 1 Processor -- start 2024-04-26T22:27:19.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.040+0000 7f06b7c7e640 1 -- start start 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b7c7e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f06b00d3820 0x7f06b00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b7c7e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06b00b4ba0 0x7f06b00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b7c7e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b00d0210 con 0x7f06b0162b40 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b7c7e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f06b00d41c0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b7c7e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f06b00d04f0 con 0x7f06b00d3820 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b621b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06b00b4ba0 0x7f06b00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b5a1a640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f06b0162b40 0x7f06b00d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56372/0 (socket says 172.21.15.62:56372) 2024-04-26T22:27:19.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b621b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f06b00b4ba0 0x7f06b00b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35380/0 (socket says 172.21.15.62:35380) 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.041+0000 7f06b5a1a640 1 -- 172.21.15.62:0/2004321361 learned_addr learned my addr 172.21.15.62:0/2004321361 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b5219640 1 --2- 172.21.15.62:0/2004321361 >> v2:172.21.15.174:3300/0 conn(0x7f06b00d3820 0x7f06b00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b621b640 1 -- 172.21.15.62:0/2004321361 >> v2:172.21.15.174:3300/0 conn(0x7f06b00d3820 msgr2=0x7f06b00d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b621b640 1 --2- 172.21.15.62:0/2004321361 >> v2:172.21.15.174:3300/0 conn(0x7f06b00d3820 0x7f06b00d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b621b640 1 -- 172.21.15.62:0/2004321361 >> v1:172.21.15.62:6789/0 conn(0x7f06b0162b40 legacy=0x7f06b00d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b4a18640 1 -- 172.21.15.62:0/2004321361 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871890973 0 0) 0x7f06b00d0210 con 0x7f06b0162b40 2024-04-26T22:27:19.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b621b640 1 -- 172.21.15.62:0/2004321361 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b00d07d0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.042+0000 7f06b621b640 1 --2- 172.21.15.62:0/2004321361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00b5030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06b008f8e0 tx=0x7f06ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467f3fd2fe72bae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b4a18640 1 -- 172.21.15.62:0/2004321361 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac007740 con 0x7f06b00b4ba0 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b4a18640 1 -- 172.21.15.62:0/2004321361 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac0078e0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b4a18640 1 -- 172.21.15.62:0/2004321361 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac007cb0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 -- 172.21.15.62:0/2004321361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 msgr2=0x7f06b00b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/2004321361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00b5030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06b008f8e0 tx=0x7f06ac02a680 comp rx=0 tx=0).stop 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 -- 172.21.15.62:0/2004321361 shutdown_connections 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/2004321361 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00b5030 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/2004321361 >> v2:172.21.15.174:3300/0 conn(0x7f06b00d3820 0x7f06b00d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 -- 172.21.15.62:0/2004321361 >> 172.21.15.62:0/2004321361 conn(0x7f06b009e550 msgr2=0x7f06b00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 -- 172.21.15.62:0/2004321361 shutdown_connections 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.043+0000 7f06b7c7e640 1 -- 172.21.15.62:0/2004321361 wait complete. 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.044+0000 7f06b7c7e640 1 Processor -- start 2024-04-26T22:27:19.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.044+0000 7f06b7c7e640 1 -- start start 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b00d3820 0x7f06b0151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 0x7f06b00b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00d12b0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06b00d0d90 con 0x7f06b0162b40 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b7c7e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f06b00cfe30 con 0x7f06b00d3820 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35384/0 (socket says 172.21.15.62:35384) 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 -- 172.21.15.62:0/487823619 learned_addr learned my addr 172.21.15.62:0/487823619 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5219640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b00d3820 0x7f06b0151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b621b640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 0x7f06b00b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 -- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b00d3820 msgr2=0x7f06b0151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b00d3820 0x7f06b0151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 -- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 msgr2=0x7f06b00b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 0x7f06b00b8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b5a1a640 1 -- 172.21.15.62:0/487823619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b00b9270 con 0x7f06b00b4ba0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.045+0000 7f06b621b640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 0x7f06b00b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06b5a1a640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c4d0 tx=0x7f06a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8018070 con 0x7f06b00b4ba0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a8007500 con 0x7f06b00b4ba0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8004a20 con 0x7f06b00b4ba0 2024-04-26T22:27:19.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b00cc440 con 0x7f06b00b4ba0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.046+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b00cca00 con 0x7f06b00b4ba0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.047+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b006a360 con 0x7f06b00b4ba0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.047+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06a8004bc0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.048+0000 7f06a67fc640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.048+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06a808adb0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.048+0000 7f06b5219640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.050 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.049+0000 7f06b5219640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006fd0 tx=0x7f06a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.052 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.052+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06a8059270 con 0x7f06b00b4ba0 2024-04-26T22:27:19.217 INFO:tasks.ceph.mds.c.smithi062.stderr:2024-04-26T22:27:19.216+0000 7f29a69c4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:27:19.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.237+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06b00ccbd0 con 0x7f06b00b4ba0 2024-04-26T22:27:19.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.238+0000 7f06a67fc640 1 -- 172.21.15.62:0/487823619 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f06a8058b70 con 0x7f06b00b4ba0 2024-04-26T22:27:19.238 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:19.239 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 msgr2=0x7f069406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006fd0 tx=0x7f06a0008040 comp rx=0 tx=0).stop 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 msgr2=0x7f06b00c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c4d0 tx=0x7f06a800c990 comp rx=0 tx=0).stop 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 shutdown_connections 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b0162b40 0x7f06b00b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b00d3820 0x7f06b0151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 --2- 172.21.15.62:0/487823619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06b00b4ba0 0x7f06b00c9630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 >> 172.21.15.62:0/487823619 conn(0x7f06b009e550 msgr2=0x7f06b00b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 shutdown_connections 2024-04-26T22:27:19.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.240+0000 7f06b7c7e640 1 -- 172.21.15.62:0/487823619 wait complete. 2024-04-26T22:27:19.251 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T22:27:19.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T22:27:19.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T22:27:19.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T22:27:19.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T22:27:19.253 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T22:27:19.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T22:27:19.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T22:27:19.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7fdb20c6d640 1 Processor -- start 2024-04-26T22:27:19.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7fdb20c6d640 1 -- start start 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.426+0000 7f1e20862640 1 Processor -- start 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.426+0000 7f1e20862640 1 -- start start 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e20862640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1e1c162df0 0x7f1e1c1705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e20862640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1e1c06c240 0x7f1e1c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e20862640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1c16d070 con 0x7f1e1c1631d0 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e20862640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1c06cbf0 con 0x7f1e1c162df0 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e20862640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1c16fba0 con 0x7f1e1c06c240 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e1affd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1e1c1631d0 0x7f1e1c16c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56386/0 (socket says 172.21.15.62:56386) 2024-04-26T22:27:19.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e1affd640 1 -- 172.21.15.62:0/3651697441 learned_addr learned my addr 172.21.15.62:0/3651697441 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.427+0000 7f1e1b7fe640 1 --2- 172.21.15.62:0/3651697441 >> v2:172.21.15.62:3300/0 conn(0x7f1e1c162df0 0x7f1e1c1705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1bfff640 1 --2- 172.21.15.62:0/3651697441 >> v2:172.21.15.174:3300/0 conn(0x7f1e1c06c240 0x7f1e1c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1a7fc640 1 -- 172.21.15.62:0/3651697441 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063389187 0 0) 0x7f1e1c16d070 con 0x7f1e1c1631d0 2024-04-26T22:27:19.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1a7fc640 1 -- 172.21.15.62:0/3651697441 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e04003520 con 0x7f1e1c1631d0 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb20c6d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdb14054690 0x7fdb140dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb20c6d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb14053310 con 0x7fdb140df130 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb20c6d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb140535f0 con 0x7fdb1400aad0 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb20c6d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fdb140e5e40 con 0x7fdb14054690 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1b7fe640 1 -- 172.21.15.62:0/3651697441 >> v2:172.21.15.174:3300/0 conn(0x7f1e1c06c240 msgr2=0x7f1e1c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1b7fe640 1 --2- 172.21.15.62:0/3651697441 >> v2:172.21.15.174:3300/0 conn(0x7f1e1c06c240 0x7f1e1c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1b7fe640 1 -- 172.21.15.62:0/3651697441 >> v1:172.21.15.62:6789/0 conn(0x7f1e1c1631d0 legacy=0x7f1e1c16c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.428+0000 7f1e1b7fe640 1 -- 172.21.15.62:0/3651697441 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c16d350 con 0x7f1e1c162df0 2024-04-26T22:27:19.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f4d310a0640 1 Processor -- start 2024-04-26T22:27:19.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f4d310a0640 1 -- start start 2024-04-26T22:27:19.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f007d3f7640 1 Processor -- start 2024-04-26T22:27:19.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb1a59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdb140df130 0x7fdb1404fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56388/0 (socket says 172.21.15.62:56388) 2024-04-26T22:27:19.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb1a59c640 1 -- 172.21.15.62:0/707514862 learned_addr learned my addr 172.21.15.62:0/707514862 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f604ee00640 1 Processor -- start 2024-04-26T22:27:19.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645231753 0 0) 0x7fdb14053310 con 0x7fdb140df130 2024-04-26T22:27:19.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb04003520 con 0x7fdb140df130 2024-04-26T22:27:19.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f4f2d9d5640 1 Processor -- start 2024-04-26T22:27:19.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f4f2d9d5640 1 -- start start 2024-04-26T22:27:19.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f007d3f7640 1 -- start start 2024-04-26T22:27:19.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f1e1b7fe640 1 --2- 172.21.15.62:0/3651697441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c162df0 0x7f1e1c1705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c166b70 tx=0x7f1e1002c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f814e69d681112a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3792364738 0 0) 0x7fdb04003520 con 0x7fdb140df130 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 >> v2:172.21.15.174:3300/0 conn(0x7fdb14054690 msgr2=0x7fdb140dfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 --2- 172.21.15.62:0/707514862 >> v2:172.21.15.174:3300/0 conn(0x7fdb14054690 0x7fdb140dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 >> v1:172.21.15.174:6789/0 conn(0x7fdb1400aad0 legacy=0x7fdb140df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb140538d0 con 0x7fdb140df130 2024-04-26T22:27:19.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d310a0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d2c06ff00 0x7f4d2c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d310a0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c0cce60 0x7f4d2c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d310a0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2c0cf550 con 0x7f4d2c1400b0 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d310a0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f4d2c143900 con 0x7f4d2c06ff00 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d310a0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c070890 con 0x7f4d2c0cce60 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f1e1a7fc640 1 -- 172.21.15.62:0/3651697441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10038070 con 0x7f1e1c162df0 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f1e1a7fc640 1 -- 172.21.15.62:0/3651697441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1000ae50 con 0x7f1e1c162df0 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f1e1a7fc640 1 -- 172.21.15.62:0/3651697441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000a700 con 0x7f1e1c162df0 2024-04-26T22:27:19.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4f2d9d5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4f20048f60 0x7f4f20006b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4f2d9d5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4f2d9d5640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f20050b90 con 0x7f4f200e7740 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4f2d9d5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4f20050930 con 0x7f4f20048f60 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4f2d9d5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200074a0 con 0x7f4f20007070 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d2a59c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d2c06ff00 0x7f4d2c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d2a59c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4d2c06ff00 0x7f4d2c070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35392/0 (socket says 172.21.15.62:35392) 2024-04-26T22:27:19.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7f4d2a59c640 1 -- 172.21.15.62:0/2941177964 learned_addr learned my addr 172.21.15.62:0/2941177964 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f007d3f7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f00700e7810 0x7f0070038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f007d3f7640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0070006190 0x7f0070006620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:55922/0 (socket says 172.21.15.62:55922) 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 -- 172.21.15.62:0/2653475113 learned_addr learned my addr 172.21.15.62:0/2653475113 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f27fff640 1 --2- 172.21.15.62:0/2653475113 >> v2:172.21.15.174:3300/0 conn(0x7f4f20048f60 0x7f4f20006b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.429+0000 7f604ee00640 1 -- start start 2024-04-26T22:27:19.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f007d3f7640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700e7bf0 con 0x7f00700eb2f0 2024-04-26T22:27:19.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f007d3f7640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0070057f10 con 0x7f00700e7810 2024-04-26T22:27:19.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f007d3f7640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f00700581f0 con 0x7f0070006190 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ee00640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f604815b270 0x7f60480b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ee00640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60480b55f0 0x7f60480d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ee00640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60480b5b70 con 0x7f60480cf720 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ee00640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f60480b5e50 con 0x7f60480b55f0 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ee00640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f60480cec40 con 0x7f604815b270 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60480b55f0 0x7f60480d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f60480b55f0 0x7f60480d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35402/0 (socket says 172.21.15.62:35402) 2024-04-26T22:27:19.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f00777fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0070006190 0x7f0070006620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 -- 172.21.15.62:0/994792756 learned_addr learned my addr 172.21.15.62:0/994792756 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604ddfe640 1 --2- 172.21.15.62:0/994792756 >> v2:172.21.15.174:3300/0 conn(0x7f604815b270 0x7f60480b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f00777fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0070006190 0x7f0070006620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59178/0 (socket says 172.21.15.62:59178) 2024-04-26T22:27:19.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f00777fe640 1 -- 172.21.15.62:0/2823455579 learned_addr learned my addr 172.21.15.62:0/2823455579 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4d2b59e640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c0cce60 0x7f4d2c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f1e20862640 1 -- 172.21.15.62:0/3651697441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c162df0 msgr2=0x7f1e1c1705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f1e20862640 1 --2- 172.21.15.62:0/3651697441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c162df0 0x7f1e1c1705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c166b70 tx=0x7f1e1002c6d0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7fce8e595640 1 Processor -- start 2024-04-26T22:27:19.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 -- 172.21.15.62:0/2653475113 >> v2:172.21.15.174:3300/0 conn(0x7f4f20048f60 msgr2=0x7f4f20006b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 --2- 172.21.15.62:0/2653475113 >> v2:172.21.15.174:3300/0 conn(0x7f4f20048f60 0x7f4f20006b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 -- 172.21.15.62:0/2653475113 >> v1:172.21.15.62:6789/0 conn(0x7f4f200e7740 legacy=0x7f4f20050220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f4f2d1d4640 1 -- 172.21.15.62:0/2653475113 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f20050e70 con 0x7f4f20007070 2024-04-26T22:27:19.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f0076ffd640 1 --2- 172.21.15.62:0/2823455579 >> v2:172.21.15.62:3300/0 conn(0x7f00700e7810 0x7f0070038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f00777fe640 1 -- 172.21.15.62:0/2823455579 >> v2:172.21.15.62:3300/0 conn(0x7f00700e7810 msgr2=0x7f0070038270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f00777fe640 1 --2- 172.21.15.62:0/2823455579 >> v2:172.21.15.62:3300/0 conn(0x7f00700e7810 0x7f0070038270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f00777fe640 1 -- 172.21.15.62:0/2823455579 >> v1:172.21.15.174:6789/0 conn(0x7f00700eb2f0 legacy=0x7f00700eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f00777fe640 1 -- 172.21.15.62:0/2823455579 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00700584d0 con 0x7f0070006190 2024-04-26T22:27:19.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fdb1c050f10 con 0x7fdb140df130 2024-04-26T22:27:19.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 >> v1:172.21.15.62:6789/0 conn(0x7fdb140df130 legacy=0x7fdb1404fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb040039b0 0x7fdb04003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb04004800 0x7fdb04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb140e5e40 con 0x7fdb04004800 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04014960 con 0x7fdb040052a0 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.430+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdb04014b60 con 0x7fdb040039b0 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7fdb1a59c640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb040039b0 0x7fdb04003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 -- 172.21.15.62:0/994792756 >> v2:172.21.15.174:3300/0 conn(0x7f604815b270 msgr2=0x7f60480b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 --2- 172.21.15.62:0/994792756 >> v2:172.21.15.174:3300/0 conn(0x7f604815b270 0x7f60480b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 -- 172.21.15.62:0/994792756 >> v1:172.21.15.174:6789/0 conn(0x7f60480cf720 legacy=0x7f604806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.431+0000 7f604d5fd640 1 -- 172.21.15.62:0/994792756 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60480cef20 con 0x7f60480b55f0 2024-04-26T22:27:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb040039b0 msgr2=0x7fdb04003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb040039b0 0x7fdb04003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/707514862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb04004800 msgr2=0x7fdb04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb04004800 0x7fdb04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/707514862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb04003520 con 0x7fdb040052a0 2024-04-26T22:27:19.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4f2d1d4640 1 --2- 172.21.15.62:0/2653475113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009870 tx=0x7f4f1c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81761b375c13533c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/707514862 reap_dead start 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f604d5fd640 1 --2- 172.21.15.62:0/994792756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60480b55f0 0x7f60480d0170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60480ce3b0 tx=0x7f604002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1c17fe9468da70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f4d2a59c640 1 -- 172.21.15.62:0/2941177964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c0cce60 msgr2=0x7f4d2c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c0cce60 0x7f4d2c0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f4d2a59c640 1 -- 172.21.15.62:0/2941177964 >> v1:172.21.15.62:6789/0 conn(0x7f4d2c1400b0 legacy=0x7f4d2c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f4d2a59c640 1 -- 172.21.15.62:0/2941177964 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c15d980 con 0x7f4d2c06ff00 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7f4d29d9b640 1 -- 172.21.15.62:0/2941177964 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860098382 0 0) 0x7f4d1c001110 con 0x7f4d2c1400b0 2024-04-26T22:27:19.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.432+0000 7fce8e595640 1 -- start start 2024-04-26T22:27:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4f277fe640 1 -- 172.21.15.62:0/2653475113 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c037070 con 0x7f4f20007070 2024-04-26T22:27:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f00777fe640 1 --2- 172.21.15.62:0/2823455579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0070006190 0x7f0070006620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f007003bf50 tx=0x7f006c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d19cedbde522829 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f1e20862640 1 -- 172.21.15.62:0/3651697441 shutdown_connections 2024-04-26T22:27:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f1e20862640 1 --2- 172.21.15.62:0/3651697441 >> v2:172.21.15.174:3300/0 conn(0x7f1e1c06c240 0x7f1e1c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f1e20862640 1 --2- 172.21.15.62:0/3651697441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c162df0 0x7f1e1c1705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f1e20862640 1 -- 172.21.15.62:0/3651697441 >> 172.21.15.62:0/3651697441 conn(0x7f1e1c0b2890 msgr2=0x7f1e1c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 0x7f4d2c070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009e50 tx=0x7f4d2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07d2d3fb259e8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4f277fe640 1 -- 172.21.15.62:0/2653475113 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f1c032440 con 0x7f4f20007070 2024-04-26T22:27:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4f277fe640 1 -- 172.21.15.62:0/2653475113 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c037070 con 0x7f4f20007070 2024-04-26T22:27:19.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb0400b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009860 tx=0x7fdb04005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd7a97b0811300a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8e595640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8e595640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fce881643a0 0x7fce8806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8e595640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce88167f70 con 0x7fce880c4500 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8e595640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fce880d0930 con 0x7fce881643a0 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8e595640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce88161520 con 0x7fce88161140 2024-04-26T22:27:19.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f0075ffb640 1 -- 172.21.15.62:0/2823455579 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c037070 con 0x7f0070006190 2024-04-26T22:27:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c009460 con 0x7fdb040052a0 2024-04-26T22:27:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f0075ffb640 1 -- 172.21.15.62:0/2823455579 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006c032440 con 0x7f0070006190 2024-04-26T22:27:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f0075ffb640 1 -- 172.21.15.62:0/2823455579 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c032810 con 0x7f0070006190 2024-04-26T22:27:19.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4d29d9b640 1 -- 172.21.15.62:0/2941177964 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24036070 con 0x7f4d2c06ff00 2024-04-26T22:27:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0c004240 con 0x7fdb040052a0 2024-04-26T22:27:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8cd92640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fce880c4500 0x7fce88163c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:52966/0 (socket says 172.21.15.62:52966) 2024-04-26T22:27:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fce8cd92640 1 -- 172.21.15.62:0/1350590889 learned_addr learned my addr 172.21.15.62:0/1350590889 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7fdb1959a640 1 -- 172.21.15.62:0/707514862 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c02a880 con 0x7fdb040052a0 2024-04-26T22:27:19.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f4d29d9b640 1 -- 172.21.15.62:0/2941177964 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d24031600 con 0x7f4d2c06ff00 2024-04-26T22:27:19.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8d593640 1 --2- 172.21.15.62:0/1350590889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f2e66d2b640 1 Processor -- start 2024-04-26T22:27:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8dd94640 1 --2- 172.21.15.62:0/1350590889 >> v2:172.21.15.174:3300/0 conn(0x7fce881643a0 0x7fce8806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391793862 0 0) 0x7fce88167f70 con 0x7fce880c4500 2024-04-26T22:27:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce74003520 con 0x7fce880c4500 2024-04-26T22:27:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f2e66d2b640 1 -- start start 2024-04-26T22:27:19.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 <== mon.1 v1:172.21.15.174:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 546546046 0 0) 0x7fce74003520 con 0x7fce880c4500 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2653475113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 msgr2=0x7f4f200538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 --2- 172.21.15.62:0/2653475113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009870 tx=0x7f4f1c02a250 comp rx=0 tx=0).stop 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8d593640 1 -- 172.21.15.62:0/1350590889 >> v2:172.21.15.174:3300/0 conn(0x7fce881643a0 msgr2=0x7fce8806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8d593640 1 --2- 172.21.15.62:0/1350590889 >> v2:172.21.15.174:3300/0 conn(0x7fce881643a0 0x7fce8806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8d593640 1 -- 172.21.15.62:0/1350590889 >> v1:172.21.15.174:6789/0 conn(0x7fce880c4500 legacy=0x7fce88163c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7fce8d593640 1 -- 172.21.15.62:0/1350590889 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce881671b0 con 0x7fce88161140 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f604cdfc640 1 -- 172.21.15.62:0/994792756 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040036070 con 0x7f60480b55f0 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f604cdfc640 1 -- 172.21.15.62:0/994792756 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6040031600 con 0x7f60480b55f0 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.433+0000 7f604cdfc640 1 -- 172.21.15.62:0/994792756 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400319d0 con 0x7f60480b55f0 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f604ee00640 1 -- 172.21.15.62:0/994792756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60480b55f0 msgr2=0x7f60480d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f604ee00640 1 --2- 172.21.15.62:0/994792756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60480b55f0 0x7f60480d0170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60480ce3b0 tx=0x7f604002a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2653475113 shutdown_connections 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 --2- 172.21.15.62:0/2653475113 >> v2:172.21.15.174:3300/0 conn(0x7f4f20048f60 0x7f4f20006b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 --2- 172.21.15.62:0/2653475113 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f20007070 0x7f4f200538a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2653475113 >> 172.21.15.62:0/2653475113 conn(0x7f4f20055bd0 msgr2=0x7f4f20055fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 -- 172.21.15.62:0/2823455579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0070006190 msgr2=0x7f0070006620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 --2- 172.21.15.62:0/2823455579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0070006190 0x7f0070006620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f007003bf50 tx=0x7f006c02a230 comp rx=0 tx=0).stop 2024-04-26T22:27:19.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4d29d9b640 1 -- 172.21.15.62:0/2941177964 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24031a10 con 0x7f4d2c06ff00 2024-04-26T22:27:19.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 -- 172.21.15.62:0/2823455579 shutdown_connections 2024-04-26T22:27:19.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 --2- 172.21.15.62:0/2823455579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0070006190 0x7f0070006620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 --2- 172.21.15.62:0/2823455579 >> v2:172.21.15.62:3300/0 conn(0x7f00700e7810 0x7f0070038270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f007d3f7640 1 -- 172.21.15.62:0/2823455579 >> 172.21.15.62:0/2823455579 conn(0x7f00700559d0 msgr2=0x7f0070055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4d310a0640 1 -- 172.21.15.62:0/2941177964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 msgr2=0x7f4d2c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f4d310a0640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 0x7f4d2c070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009e50 tx=0x7f4d2402a610 comp rx=0 tx=0).stop 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 -- 172.21.15.62:0/2941177964 shutdown_connections 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 0x7f4d2c070350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 --2- 172.21.15.62:0/2941177964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c0cce60 0x7f4d2c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 -- 172.21.15.62:0/2941177964 >> 172.21.15.62:0/2941177964 conn(0x7f4d2c0d0c80 msgr2=0x7f4d2c0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f1e20862640 1 -- 172.21.15.62:0/3651697441 shutdown_connections 2024-04-26T22:27:19.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.434+0000 7f1e20862640 1 -- 172.21.15.62:0/3651697441 wait complete. 2024-04-26T22:27:19.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 -- 172.21.15.62:0/2941177964 shutdown_connections 2024-04-26T22:27:19.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4d310a0640 1 -- 172.21.15.62:0/2941177964 wait complete. 2024-04-26T22:27:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f007d3f7640 1 -- 172.21.15.62:0/2823455579 shutdown_connections 2024-04-26T22:27:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f007d3f7640 1 -- 172.21.15.62:0/2823455579 wait complete. 2024-04-26T22:27:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2653475113 shutdown_connections 2024-04-26T22:27:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7fdb20c6d640 1 -- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 msgr2=0x7fdb0400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7fdb20c6d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb0400b9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009860 tx=0x7fdb04005830 comp rx=0 tx=0).stop 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f604ee00640 1 -- 172.21.15.62:0/994792756 shutdown_connections 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f604ee00640 1 --2- 172.21.15.62:0/994792756 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f60480b55f0 0x7f60480d0170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f604ee00640 1 --2- 172.21.15.62:0/994792756 >> v2:172.21.15.174:3300/0 conn(0x7f604815b270 0x7f60480b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f604ee00640 1 -- 172.21.15.62:0/994792756 >> 172.21.15.62:0/994792756 conn(0x7f60480b2890 msgr2=0x7f60480b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7fdb20c6d640 1 -- 172.21.15.62:0/707514862 shutdown_connections 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7fdb20c6d640 1 --2- 172.21.15.62:0/707514862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb0400b9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7fdb20c6d640 1 -- 172.21.15.62:0/707514862 >> 172.21.15.62:0/707514862 conn(0x7fdb14009fe0 msgr2=0x7fdb1400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e66d2b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e600cfa90 0x7f2e60072070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e66d2b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2e600cfee0 0x7f2e600725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e66d2b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e6006e6c0 con 0x7f2e6014d2e0 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e66d2b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6006e9a0 con 0x7f2e600cfa90 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e66d2b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2e6006ec80 con 0x7f2e600cfee0 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e5ffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e600cfa90 0x7f2e60072070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e5ffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2e600cfa90 0x7f2e60072070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35430/0 (socket says 172.21.15.62:35430) 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f2e5ffff640 1 -- 172.21.15.62:0/3018068391 learned_addr learned my addr 172.21.15.62:0/3018068391 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.435+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2653475113 wait complete. 2024-04-26T22:27:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f2e652c8640 1 --2- 172.21.15.62:0/3018068391 >> v2:172.21.15.174:3300/0 conn(0x7f2e600cfee0 0x7f2e600725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7fdb20c6d640 1 -- 172.21.15.62:0/707514862 shutdown_connections 2024-04-26T22:27:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f604ee00640 1 -- 172.21.15.62:0/994792756 shutdown_connections 2024-04-26T22:27:19.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f604ee00640 1 -- 172.21.15.62:0/994792756 wait complete. 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f1e20862640 1 Processor -- start 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f2e5ffff640 1 -- 172.21.15.62:0/3018068391 >> v2:172.21.15.174:3300/0 conn(0x7f2e600cfee0 msgr2=0x7f2e600725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f2e5ffff640 1 --2- 172.21.15.62:0/3018068391 >> v2:172.21.15.174:3300/0 conn(0x7f2e600cfee0 0x7f2e600725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f2e5ffff640 1 -- 172.21.15.62:0/3018068391 >> v1:172.21.15.174:6789/0 conn(0x7f2e6014d2e0 legacy=0x7f2e600b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f2e5ffff640 1 -- 172.21.15.62:0/3018068391 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6006ef60 con 0x7f2e600cfa90 2024-04-26T22:27:19.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7f1e20862640 1 -- start start 2024-04-26T22:27:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f007d3f7640 1 Processor -- start 2024-04-26T22:27:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f007d3f7640 1 -- start start 2024-04-26T22:27:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f4f2d9d5640 1 Processor -- start 2024-04-26T22:27:19.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.436+0000 7fdb20c6d640 1 -- 172.21.15.62:0/707514862 wait complete. 2024-04-26T22:27:19.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f2e5ffff640 1 --2- 172.21.15.62:0/3018068391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e600cfa90 0x7f2e60072070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e60070540 tx=0x7f2e5402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647135957726acfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f4f2d9d5640 1 -- start start 2024-04-26T22:27:19.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7fce8d593640 1 --2- 172.21.15.62:0/1350590889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880c3fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce780097a0 tx=0x7fce7802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9d5c24e5a3bc620 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f4d310a0640 1 Processor -- start 2024-04-26T22:27:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e20862640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e1c06c240 0x7f1e1c164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e20862640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e1c162df0 0x7f1e1c161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 0x7f00700dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00700e7810 0x7f007004c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f4d310a0640 1 -- start start 2024-04-26T22:27:19.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00700eb2f0 0x7f00700df300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 0x7f00700dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e1c06c240 0x7f1e1c164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e1c06c240 0x7f1e1c164bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59200/0 (socket says 172.21.15.62:59200) 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00700ebe80 con 0x7f0070006190 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f0076ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 0x7f00700dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35442/0 (socket says 172.21.15.62:35442) 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f0076ffd640 1 -- 172.21.15.62:0/299744188 learned_addr learned my addr 172.21.15.62:0/299744188 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00700ec190 con 0x7f00700eb2f0 2024-04-26T22:27:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e1b7fe640 1 -- 172.21.15.62:0/1950679858 learned_addr learned my addr 172.21.15.62:0/1950679858 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0070058cd0 con 0x7f00700e7810 2024-04-26T22:27:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f1e20862640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c1631d0 0x7f1e1c16e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f00767fc640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00700e7810 0x7f007004c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f4f2d9d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20007070 0x7f4f2004f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f2e5f7fe640 1 -- 172.21.15.62:0/3018068391 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54037070 con 0x7f2e600cfa90 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f2e5f7fe640 1 -- 172.21.15.62:0/3018068391 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e54032440 con 0x7f2e600cfa90 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.437+0000 7f2e5f7fe640 1 -- 172.21.15.62:0/3018068391 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54032810 con 0x7f2e600cfa90 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f2e66d2b640 1 -- 172.21.15.62:0/3018068391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e600cfa90 msgr2=0x7f2e60072070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/3018068391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e600cfa90 0x7f2e60072070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e60070540 tx=0x7f2e5402a230 comp rx=0 tx=0).stop 2024-04-26T22:27:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 -- 172.21.15.62:0/3018068391 shutdown_connections 2024-04-26T22:27:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/3018068391 >> v2:172.21.15.174:3300/0 conn(0x7f2e600cfee0 0x7f2e600725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/3018068391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e600cfa90 0x7f2e60072070 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2e60070540 tx=0x7f2e5402a230 comp rx=0 tx=0).stop 2024-04-26T22:27:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c16ccc0 con 0x7f1e1c1631d0 2024-04-26T22:27:19.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 -- 172.21.15.62:0/3018068391 >> 172.21.15.62:0/3018068391 conn(0x7f2e600d0d50 msgr2=0x7f2e600d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1700d0 con 0x7f1e1c06c240 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1c162b10 con 0x7f1e1c162df0 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7fdb20c6d640 1 Processor -- start 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7fdb20c6d640 1 -- start start 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1affd640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e1c162df0 0x7f1e1c161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f00777fe640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00700eb2f0 0x7f00700df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78037070 con 0x7fce88161140 2024-04-26T22:27:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7802ab10 con 0x7fce88161140 2024-04-26T22:27:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fce7ffff640 1 -- 172.21.15.62:0/1350590889 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce800019f0 con 0x7fce88161140 2024-04-26T22:27:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c1631d0 0x7f1e1c16e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 -- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00700e7810 msgr2=0x7f007004c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2d9d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20048f60 0x7f4f201804f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f00700e7810 0x7f007004c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2d9d5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f200e7740 0x7f4f20186500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2d9d5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20051710 con 0x7f4f20007070 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 -- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00700eb2f0 msgr2=0x7f00700df300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2d9d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20051430 con 0x7f4f20048f60 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2c9d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20007070 0x7f4f2004f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2d9d5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4f200e7b20 con 0x7f4f200e7740 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2c9d3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20007070 0x7f4f2004f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35458/0 (socket says 172.21.15.62:35458) 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f4f2c9d3640 1 -- 172.21.15.62:0/2922268629 learned_addr learned my addr 172.21.15.62:0/2922268629 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 -- 172.21.15.62:0/3018068391 shutdown_connections 2024-04-26T22:27:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00700eb2f0 0x7f00700df300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070051290 con 0x7f0070006190 2024-04-26T22:27:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f0076ffd640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 0x7f00700dbdd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f006800d800 tx=0x7f006800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fdb20c6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb140d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fdb20c6d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb140d1300 0x7fdb140d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800b6c0 con 0x7f0070006190 2024-04-26T22:27:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f604ee00640 1 Processor -- start 2024-04-26T22:27:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0070051570 con 0x7f0070006190 2024-04-26T22:27:19.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0070051b30 con 0x7f0070006190 2024-04-26T22:27:19.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7fdb20c6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 0x7fdb140e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f2e66d2b640 1 -- 172.21.15.62:0/3018068391 wait complete. 2024-04-26T22:27:19.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0068019070 con 0x7f0070006190 2024-04-26T22:27:19.481 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800ba80 con 0x7f0070006190 2024-04-26T22:27:19.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.438+0000 7f604ee00640 1 -- start start 2024-04-26T22:27:19.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb1a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb140d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb1ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 0x7fdb140e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb1ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 0x7fdb140e2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35476/0 (socket says 172.21.15.62:35476) 2024-04-26T22:27:19.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 0x7f6048153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb1a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb140d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59232/0 (socket says 172.21.15.62:59232) 2024-04-26T22:27:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/3132827692 learned_addr learned my addr 172.21.15.62:0/3132827692 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb20c6d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14054170 con 0x7fdb140d7910 2024-04-26T22:27:19.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb20c6d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14053e90 con 0x7fdb040052a0 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60480cf720 0x7f604815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604815b270 0x7f60480d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ddfe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 0x7f6048153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ddfe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 0x7f6048153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:55952/0 (socket says 172.21.15.62:55952) 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ddfe640 1 -- 172.21.15.62:0/721963750 learned_addr learned my addr 172.21.15.62:0/721963750 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f604815b7d0 con 0x7f604815b270 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f604815b650 con 0x7f60480cf720 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6048073b80 con 0x7f60480b55f0 2024-04-26T22:27:19.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f604e5ff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604815b270 0x7f60480d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f604d5fd640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60480cf720 0x7f604815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 0x7f4d2c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d2c0cce60 0x7f4d2c133190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 0x7f4d2c153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c15e220 con 0x7f4d2c06ff00 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c15df40 con 0x7f4d2c0cce60 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d310a0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2c142b10 con 0x7f4d2c1400b0 2024-04-26T22:27:19.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb20c6d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdb140e04e0 con 0x7fdb140d1300 2024-04-26T22:27:19.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4f2d1d4640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f200e7740 0x7f4f20186500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d2b59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 0x7f4d2c153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20048f60 0x7f4f201804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7fdb19d9b640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb140d1300 0x7fdb140d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d2b59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 0x7f4d2c153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:55988/0 (socket says 172.21.15.62:55988) 2024-04-26T22:27:19.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.440+0000 7f4d2b59e640 1 -- 172.21.15.62:0/1213540052 learned_addr learned my addr 172.21.15.62:0/1213540052 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d2c0cce60 0x7f4d2c133190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 -- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e1c162df0 msgr2=0x7f1e1c161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e1c162df0 0x7f1e1c161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 -- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e1c06c240 msgr2=0x7f1e1c164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e1c06c240 0x7f1e1c164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.439+0000 7f1e1bfff640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c1652b0 con 0x7f1e1c1631d0 2024-04-26T22:27:19.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 -- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60480cf720 msgr2=0x7f604815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 -- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f200e7740 msgr2=0x7f4f20186500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4f200e7740 0x7f4f20186500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 -- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20007070 msgr2=0x7f4f2004f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4f20007070 0x7f4f2004f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f20186d60 con 0x7f4f20048f60 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 -- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d2c0cce60 msgr2=0x7f4d2c133190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 -- 172.21.15.62:0/1350590889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 msgr2=0x7fce880c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 --2- 172.21.15.62:0/1350590889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880c3fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce780097a0 tx=0x7fce7802a080 comp rx=0 tx=0).stop 2024-04-26T22:27:19.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f1e1bfff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c1631d0 0x7f1e1c16e730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e14007d90 tx=0x7f1e1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d2c0cce60 0x7f4d2c133190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 -- 172.21.15.62:0/1213540052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 msgr2=0x7f4d2c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 -- 172.21.15.62:0/1350590889 shutdown_connections 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 --2- 172.21.15.62:0/1350590889 >> v2:172.21.15.174:3300/0 conn(0x7fce881643a0 0x7fce8806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 --2- 172.21.15.62:0/1350590889 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880c3fc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fce8e595640 1 -- 172.21.15.62:0/1350590889 >> 172.21.15.62:0/1350590889 conn(0x7fce880b2b80 msgr2=0x7fce880b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d2c06ff00 0x7f4d2c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0d0860 con 0x7f4d2c1400b0 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d2c0cce60 0x7f4d2c133190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60480cf720 0x7f604815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 -- 172.21.15.62:0/721963750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604815b270 msgr2=0x7f60480d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604815b270 0x7f60480d1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60481537b0 con 0x7f60480b55f0 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604e5ff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f604815b270 0x7f60480d1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f2e66d2b640 1 Processor -- start 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f2e66d2b640 1 -- start start 2024-04-26T22:27:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f604ddfe640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 0x7f6048153270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6044007ab0 tx=0x7f6044007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f27fff640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20048f60 0x7f4f201804f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400d520 tx=0x7f4f1400d920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6044019070 con 0x7f60480b55f0 2024-04-26T22:27:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f604400a9e0 con 0x7f60480b55f0 2024-04-26T22:27:19.493 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4d2b59e640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 0x7f4d2c153960 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e820 tx=0x7f4d2000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604400ad90 con 0x7f60480b55f0 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb140d1300 msgr2=0x7fdb140d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb140d1300 0x7fdb140d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 msgr2=0x7fdb140d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb040052a0 0x7fdb140d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb140d6870 con 0x7fdb140d7910 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.441+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140125f0 con 0x7f4f20048f60 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f14019070 con 0x7f4f20048f60 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140129d0 con 0x7f4f20048f60 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7fdb1ad9d640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 0x7fdb140e2db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000c00 tx=0x7fdb0c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20017070 con 0x7f4d2c1400b0 2024-04-26T22:27:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2000ee70 con 0x7f4d2c1400b0 2024-04-26T22:27:19.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f20187040 con 0x7f4f20048f60 2024-04-26T22:27:19.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f20186a40 con 0x7f4f20048f60 2024-04-26T22:27:19.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c040070 con 0x7fdb140d7910 2024-04-26T22:27:19.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb0c0035f0 con 0x7fdb140d7910 2024-04-26T22:27:19.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2000cb60 con 0x7f4d2c1400b0 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c1326b0 con 0x7f4d2c1400b0 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c133a80 con 0x7f4d2c1400b0 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f2e66d2b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e600cfee0 0x7f2e602017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f2e66d2b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e6014d2e0 0x7f2e601ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400ed70 con 0x7f1e1c1631d0 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e14003990 con 0x7f1e1c1631d0 2024-04-26T22:27:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14003d80 con 0x7f1e1c1631d0 2024-04-26T22:27:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e6014d2e0 0x7f2e601ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e6014d2e0 0x7f2e601ebf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35490/0 (socket says 172.21.15.62:35490) 2024-04-26T22:27:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e5ffff640 1 -- 172.21.15.62:0/2025845766 learned_addr learned my addr 172.21.15.62:0/2025845766 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e601ec4d0 0x7f2e60205580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c165590 con 0x7f1e1c1631d0 2024-04-26T22:27:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c165b50 con 0x7f1e1c1631d0 2024-04-26T22:27:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d2c0ce5b0 con 0x7f4d2c1400b0 2024-04-26T22:27:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7fce8e595640 1 -- 172.21.15.62:0/1350590889 shutdown_connections 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6006e250 con 0x7f2e6014d2e0 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e600b4490 con 0x7f2e600cfee0 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6006e3d0 con 0x7f2e601ec4d0 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f2e652c8640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e601ec4d0 0x7f2e60205580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6048159a50 con 0x7f60480b55f0 2024-04-26T22:27:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f604815a010 con 0x7f60480b55f0 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.442+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f006800bc20 con 0x7f0070006190 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0058001030 con 0x7f0070006190 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f005f7fe640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f006406c8f0 0x7f006406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f006808c070 con 0x7f0070006190 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f005f7fe640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] conn(0x7f0064074af0 0x7f0064076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f00640775c0 con 0x7f0064074af0 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f00767fc640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f006406c8f0 0x7f006406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 -- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e600cfee0 msgr2=0x7f2e602017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f00777fe640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] conn(0x7f0064074af0 0x7f0064076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e64ac7640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e600cfee0 0x7f2e602017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e600cfee0 0x7f2e602017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 -- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e6014d2e0 msgr2=0x7f2e601ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e6014d2e0 0x7f2e601ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e60143ed0 con 0x7f2e601ec4d0 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e64ac7640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e600cfee0 0x7f2e602017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1dfc001030 con 0x7f1e1c1631d0 2024-04-26T22:27:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0039e0 con 0x7fdb140d7910 2024-04-26T22:27:19.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f604815a3f0 con 0x7f60480b55f0 2024-04-26T22:27:19.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f2e652c8640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e601ec4d0 0x7f2e60205580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e58007d90 tx=0x7f2e5800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.443+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f20180a30 con 0x7f4f20048f60 2024-04-26T22:27:19.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.444+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f14012b70 con 0x7f4f20048f60 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7fdb20c6d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb140d6b50 con 0x7fdb140d7910 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7fdb20c6d640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb140d7110 con 0x7fdb140d7910 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f257fa640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4f0c06c920 0x7f4f0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4f1408be40 con 0x7f4f20048f60 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f2c9d3640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4f0c06c920 0x7f4f0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f257fa640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] conn(0x7f4f0c074ac0 0x7f4f0c076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f14009e50 con 0x7f4f0c074ac0 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4f1408c2d0 con 0x7f4f20048f60 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4f2d1d4640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] conn(0x7f4f0c074ac0 0x7f4f0c076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0068055fa0 con 0x7f0070006190 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f00767fc640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f006406c8f0 0x7f006406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0078051470 tx=0x7f007806a160 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f00777fe640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] conn(0x7f0064074af0 0x7f0064076f10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800edd0 con 0x7f2e601ec4d0 2024-04-26T22:27:19.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e58003990 con 0x7f2e601ec4d0 2024-04-26T22:27:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58003d80 con 0x7f2e601ec4d0 2024-04-26T22:27:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e601441b0 con 0x7f2e601ec4d0 2024-04-26T22:27:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e60144770 con 0x7f2e601ec4d0 2024-04-26T22:27:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2001a070 con 0x7f4d2c1400b0 2024-04-26T22:27:19.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4f2d1d4640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] conn(0x7f4f0c074ac0 0x7f4f0c076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4d137fe640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d1806c9c0 0x7f4d1806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d2008c1b0 con 0x7f4d2c1400b0 2024-04-26T22:27:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb140d1040 con 0x7fdb140d7910 2024-04-26T22:27:19.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4d137fe640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] conn(0x7f4d18074c40 0x7f4d18077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f604401c070 con 0x7f60480b55f0 2024-04-26T22:27:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d18077710 con 0x7f4d18074c40 2024-04-26T22:27:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4d2008c5a0 con 0x7f4d2c1400b0 2024-04-26T22:27:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f4d2ad9d640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d1806c9c0 0x7f4d1806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.509 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] conn(0x7f4d18074c40 0x7f4d18077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e60144b50 con 0x7f2e601ec4d0 2024-04-26T22:27:19.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f603e7fc640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f602c06c8f0 0x7f602c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f604408be90 con 0x7f60480b55f0 2024-04-26T22:27:19.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f603e7fc640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] conn(0x7f602c074af0 0x7f602c076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f4d2a59c640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] conn(0x7f4d18074c40 0x7f4d18077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f602c0775c0 con 0x7f602c074af0 2024-04-26T22:27:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.445+0000 7fce8e595640 1 -- 172.21.15.62:0/1350590889 wait complete. 2024-04-26T22:27:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7fce8e595640 1 Processor -- start 2024-04-26T22:27:19.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7fce8e595640 1 -- start start 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.446+0000 7f4f2c9d3640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4f0c06c920 0x7f4f0c06ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f28000b60 tx=0x7f4f280660f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== osd.3 v2:172.21.15.62:6808/3538853986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f14009e50 con 0x7f4f0c074ac0 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e5801b020 con 0x7f2e601ec4d0 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f604408c280 con 0x7f60480b55f0 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f604e5ff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] conn(0x7f602c074af0 0x7f602c076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f604d5fd640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f602c06c8f0 0x7f602c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb0c043070 con 0x7fdb140d7910 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f604e5ff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] conn(0x7f602c074af0 0x7f602c076f10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e1401b020 con 0x7f1e1c1631d0 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f1e03fff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e0806c8f0 0x7f1e0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e1408bac0 con 0x7f1e1c1631d0 2024-04-26T22:27:19.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f1e03fff640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] conn(0x7f1e08074a90 0x7f1e08076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.447+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e14012e50 con 0x7f1e08074a90 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f1e1b7fe640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e0806c8f0 0x7f1e0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f1e1affd640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] conn(0x7f1e08074a90 0x7f1e08076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1e14055b40 con 0x7f1e1c1631d0 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f2e5d7fa640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e4406c970 0x7f2e4406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== osd.5 v2:172.21.15.174:6816/3260296682 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d18077710 con 0x7f4d18074c40 2024-04-26T22:27:19.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f4d2ad9d640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d1806c9c0 0x7f4d1806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000f60 tx=0x7f4d1c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e5808bb40 con 0x7f2e601ec4d0 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e5d7fa640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] conn(0x7f2e44074b90 0x7f2e44076fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e64ac7640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e4406c970 0x7f2e4406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e58012e50 con 0x7f2e44074b90 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2e5808e290 con 0x7f2e601ec4d0 2024-04-26T22:27:19.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f1e1affd640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] conn(0x7f1e08074a90 0x7f1e08076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7f1e1b7fe640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e0806c8f0 0x7f1e0806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002c710 tx=0x7f1e1000a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== osd.7 v2:172.21.15.174:6808/2710114458 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f00640775c0 con 0x7f0064074af0 2024-04-26T22:27:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== osd.6 v2:172.21.15.174:6824/341063153 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f602c0775c0 con 0x7f602c074af0 2024-04-26T22:27:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f604d5fd640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f602c06c8f0 0x7f602c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6040033000 tx=0x7f6040009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.516 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8e595640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce880c4500 0x7fce880b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.448+0000 7fdb0affd640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdafc06c9c0 0x7fdafc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb0c0aaa90 con 0x7fdb140d7910 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb0affd640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] conn(0x7fdafc074be0 0x7fdafc077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb0c001d30 con 0x7fdafc074be0 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c0af2a0 con 0x7fdb140d7910 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb1a59c640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdafc06c9c0 0x7fdafc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8e595640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8e595640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce881643a0 0x7fce880af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e5ffff640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] conn(0x7f2e44074b90 0x7f2e44076fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8e595640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce88164a30 con 0x7fce880c4500 2024-04-26T22:27:19.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fdb19d9b640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] conn(0x7fdafc074be0 0x7fdafc077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== osd.4 v2:172.21.15.174:6800/2799507902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e14012e50 con 0x7f1e08074a90 2024-04-26T22:27:19.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8e595640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce88167770 con 0x7fce881643a0 2024-04-26T22:27:19.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7f2e64ac7640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e4406c970 0x7f2e4406ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7fce8e595640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce88167a50 con 0x7fce88161140 2024-04-26T22:27:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.449+0000 7fce8cd92640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7fce8cd92640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:55992/0 (socket says 172.21.15.62:55992) 2024-04-26T22:27:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7fdb19d9b640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] conn(0x7fdafc074be0 0x7fdafc077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7fce8cd92640 1 -- 172.21.15.62:0/650450095 learned_addr learned my addr 172.21.15.62:0/650450095 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7f2e5ffff640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] conn(0x7f2e44074b90 0x7f2e44076fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.450+0000 7fce8dd94640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce881643a0 0x7fce880af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 -- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce881643a0 msgr2=0x7fce880af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce881643a0 0x7fce880af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 -- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce880c4500 msgr2=0x7fce880b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8d593640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce880c4500 0x7fce880b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce880c4500 0x7fce880b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 -- 172.21.15.62:0/650450095 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce880afb80 con 0x7fce88161140 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8dd94640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce881643a0 0x7fce880af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== osd.0 v2:172.21.15.62:6824/4028840618 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb0c001d30 con 0x7fdafc074be0 2024-04-26T22:27:19.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8d593640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce880c4500 0x7fce880b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7fce8cd92640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880aee10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce80001fb0 tx=0x7fce8000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.451+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== osd.1 v2:172.21.15.62:6814/2498719167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e58012e50 con 0x7f2e44074b90 2024-04-26T22:27:19.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.452+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80017070 con 0x7fce88161140 2024-04-26T22:27:19.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.452+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce80012400 con 0x7fce88161140 2024-04-26T22:27:19.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.452+0000 7fce8e595640 1 -- 172.21.15.62:0/650450095 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8815d230 con 0x7fce88161140 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.452+0000 7fce8e595640 1 -- 172.21.15.62:0/650450095 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fce8815d7f0 con 0x7fce88161140 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.454+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80012810 con 0x7fce88161140 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.455+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce8001a070 con 0x7fce88161140 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.456+0000 7fce7dffb640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce6c06c9c0 0x7fce6c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.456+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce8008baf0 con 0x7fce88161140 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.456+0000 7fce8d593640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce6c06c9c0 0x7fce6c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.458+0000 7fce8e595640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] conn(0x7fce68001650 0x7fce68003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.458+0000 7fce8dd94640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] conn(0x7fce68001650 0x7fce68003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.458+0000 7fce8e595640 1 -- 172.21.15.62:0/650450095 --> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce68006bb0 con 0x7fce68001650 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.458+0000 7fce8d593640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce6c06c9c0 0x7fce6c06ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce78034000 tx=0x7fce7802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.459+0000 7fce8dd94640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] conn(0x7fce68001650 0x7fce68003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.460+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== osd.2 v2:172.21.15.62:6800/1123744749 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce68006bb0 con 0x7fce68001650 2024-04-26T22:27:19.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.466+0000 7fdb1a59c640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdafc06c9c0 0x7fdafc06ee80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c063ef0 tx=0x7fdb1c0660f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.480+0000 7f4f2d9d5640 1 -- 172.21.15.62:0/2922268629 --> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f201876f0 con 0x7f4f0c074ac0 2024-04-26T22:27:19.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.480+0000 7f4f257fa640 1 -- 172.21.15.62:0/2922268629 <== osd.3 v2:172.21.15.62:6808/3538853986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f201876f0 con 0x7f4f0c074ac0 2024-04-26T22:27:19.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.481+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] conn(0x7f4f0c074ac0 msgr2=0x7f4f0c076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.481+0000 7f4f12ffd640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.62:6808/3538853986,v1:172.21.15.62:6809/3538853986] conn(0x7f4f0c074ac0 0x7f4f0c076ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.481+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4f0c06c920 msgr2=0x7f4f0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.481+0000 7f4f12ffd640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4f0c06c920 0x7f4f0c06ede0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4f28000b60 tx=0x7f4f280660f0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20048f60 msgr2=0x7f4f201804f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f4f12ffd640 1 --2- 172.21.15.62:0/2922268629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20048f60 0x7f4f201804f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400d520 tx=0x7f4f1400d920 comp rx=0 tx=0).stop 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f4f2d1d4640 1 -- 172.21.15.62:0/2922268629 reap_dead start 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 shutdown_connections 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 >> 172.21.15.62:0/2922268629 conn(0x7f4f20055bd0 msgr2=0x7f4f200ea4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.482+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 --> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0058002dd0 con 0x7f0064074af0 2024-04-26T22:27:19.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 shutdown_connections 2024-04-26T22:27:19.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f005f7fe640 1 -- 172.21.15.62:0/299744188 <== osd.7 v2:172.21.15.174:6808/2710114458 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0058002dd0 con 0x7f0064074af0 2024-04-26T22:27:19.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f4f12ffd640 1 -- 172.21.15.62:0/2922268629 wait complete. 2024-04-26T22:27:19.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 --> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d2c069b90 con 0x7f4d18074c40 2024-04-26T22:27:19.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 --> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1dfc002dd0 con 0x7f1e08074a90 2024-04-26T22:27:19.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] conn(0x7f0064074af0 msgr2=0x7f0064076f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f007d3f7640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6808/2710114458,v1:172.21.15.174:6809/2710114458] conn(0x7f0064074af0 0x7f0064076f10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f006406c8f0 msgr2=0x7f006406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.483+0000 7f007d3f7640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f006406c8f0 0x7f006406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0078051470 tx=0x7f007806a160 comp rx=0 tx=0).stop 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 msgr2=0x7f00700dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 --2- 172.21.15.62:0/299744188 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0070006190 0x7f00700dbdd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f006800d800 tx=0x7f006800dcc0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f604ee00640 1 -- 172.21.15.62:0/721963750 --> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6048076890 con 0x7f602c074af0 2024-04-26T22:27:19.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f1e03fff640 1 -- 172.21.15.62:0/1950679858 <== osd.4 v2:172.21.15.174:6800/2799507902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1dfc002dd0 con 0x7f1e08074a90 2024-04-26T22:27:19.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f4d137fe640 1 -- 172.21.15.62:0/1213540052 <== osd.5 v2:172.21.15.174:6816/3260296682 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4d2c069b90 con 0x7f4d18074c40 2024-04-26T22:27:19.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f00777fe640 1 -- 172.21.15.62:0/299744188 reap_dead start 2024-04-26T22:27:19.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 shutdown_connections 2024-04-26T22:27:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 >> 172.21.15.62:0/299744188 conn(0x7f00700559d0 msgr2=0x7f00700e6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f603e7fc640 1 -- 172.21.15.62:0/721963750 <== osd.6 v2:172.21.15.174:6824/341063153 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6048076890 con 0x7f602c074af0 2024-04-26T22:27:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 shutdown_connections 2024-04-26T22:27:19.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f007d3f7640 1 -- 172.21.15.62:0/299744188 wait complete. 2024-04-26T22:27:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] conn(0x7f4d18074c40 msgr2=0x7f4d18077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f4d310a0640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6816/3260296682,v1:172.21.15.174:6817/3260296682] conn(0x7f4d18074c40 0x7f4d18077060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.484+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] conn(0x7f1e08074a90 msgr2=0x7f1e08076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6800/2799507902,v1:172.21.15.174:6801/2799507902] conn(0x7f1e08074a90 0x7f1e08076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 --> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb140053c0 con 0x7fdafc074be0 2024-04-26T22:27:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e0806c8f0 msgr2=0x7f1e0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e0806c8f0 0x7f1e0806edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1e1002c710 tx=0x7f1e1000a000 comp rx=0 tx=0).stop 2024-04-26T22:27:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d1806c9c0 msgr2=0x7f4d1806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d1806c9c0 0x7f4d1806ee80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c000f60 tx=0x7f4d1c0079b0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c1631d0 msgr2=0x7f1e1c16e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 --2- 172.21.15.62:0/1950679858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1e1c1631d0 0x7f1e1c16e730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e14007d90 tx=0x7f1e1400e4b0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 msgr2=0x7f4d2c153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 --2- 172.21.15.62:0/1213540052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d2c1400b0 0x7f4d2c153960 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000e820 tx=0x7f4d2000ec20 comp rx=0 tx=0).stop 2024-04-26T22:27:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] conn(0x7f602c074af0 msgr2=0x7f602c076f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6824/341063153,v1:172.21.15.174:6825/341063153] conn(0x7f602c074af0 0x7f602c076f10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d2b59e640 1 -- 172.21.15.62:0/1213540052 reap_dead start 2024-04-26T22:27:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 shutdown_connections 2024-04-26T22:27:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e1bfff640 1 -- 172.21.15.62:0/1950679858 reap_dead start 2024-04-26T22:27:19.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 >> 172.21.15.62:0/1213540052 conn(0x7f4d2c0d0c80 msgr2=0x7f4d2c072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f602c06c8f0 msgr2=0x7f602c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f602c06c8f0 0x7f602c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6040033000 tx=0x7f6040009000 comp rx=0 tx=0).stop 2024-04-26T22:27:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 shutdown_connections 2024-04-26T22:27:19.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7fdb0affd640 1 -- 172.21.15.62:0/3132827692 <== osd.0 v2:172.21.15.62:6824/4028840618 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb140053c0 con 0x7fdafc074be0 2024-04-26T22:27:19.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 shutdown_connections 2024-04-26T22:27:19.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 msgr2=0x7f6048153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 --2- 172.21.15.62:0/721963750 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f60480b55f0 0x7f6048153270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6044007ab0 tx=0x7f6044007f70 comp rx=0 tx=0).stop 2024-04-26T22:27:19.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 >> 172.21.15.62:0/1950679858 conn(0x7f1e1c0b2890 msgr2=0x7f1e1c072a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f4d310a0640 1 -- 172.21.15.62:0/1213540052 wait complete. 2024-04-26T22:27:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f604e5ff640 1 -- 172.21.15.62:0/721963750 reap_dead start 2024-04-26T22:27:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.485+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 shutdown_connections 2024-04-26T22:27:19.540 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 >> 172.21.15.62:0/721963750 conn(0x7f60480b2890 msgr2=0x7f60480726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 shutdown_connections 2024-04-26T22:27:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7f1e20862640 1 -- 172.21.15.62:0/1950679858 wait complete. 2024-04-26T22:27:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 shutdown_connections 2024-04-26T22:27:19.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7f602bfff640 1 -- 172.21.15.62:0/721963750 wait complete. 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] conn(0x7fdafc074be0 msgr2=0x7fdafc077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:6824/4028840618,v1:172.21.15.62:6825/4028840618] conn(0x7fdafc074be0 0x7fdafc077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdafc06c9c0 msgr2=0x7fdafc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdafc06c9c0 0x7fdafc06ee80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c063ef0 tx=0x7fdb1c0660f0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 msgr2=0x7fdb140e2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb08ff9640 1 --2- 172.21.15.62:0/3132827692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb140d7910 0x7fdb140e2db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000c00 tx=0x7fdb0c0047d0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.486+0000 7fdb1ad9d640 1 -- 172.21.15.62:0/3132827692 reap_dead start 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.487+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 shutdown_connections 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.487+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 >> 172.21.15.62:0/3132827692 conn(0x7fdb14009fe0 msgr2=0x7fdb140d9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.487+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 shutdown_connections 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.487+0000 7fdb08ff9640 1 -- 172.21.15.62:0/3132827692 wait complete. 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.487+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 --> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e6006a360 con 0x7f2e44074b90 2024-04-26T22:27:19.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.488+0000 7f2e5d7fa640 1 -- 172.21.15.62:0/2025845766 <== osd.1 v2:172.21.15.62:6814/2498719167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2e6006a360 con 0x7f2e44074b90 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.488+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] conn(0x7f2e44074b90 msgr2=0x7f2e44076fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.62:6814/2498719167,v1:172.21.15.62:6816/2498719167] conn(0x7f2e44074b90 0x7f2e44076fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e4406c970 msgr2=0x7f2e4406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e4406c970 0x7f2e4406ee30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c009040 comp rx=0 tx=0).stop 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e601ec4d0 msgr2=0x7f2e60205580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e66d2b640 1 --2- 172.21.15.62:0/2025845766 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2e601ec4d0 0x7f2e60205580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2e58007d90 tx=0x7f2e5800e4b0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.489+0000 7f2e652c8640 1 -- 172.21.15.62:0/2025845766 reap_dead start 2024-04-26T22:27:19.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7fce8e595640 1 -- 172.21.15.62:0/650450095 --> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce68005c90 con 0x7fce68001650 2024-04-26T22:27:19.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 shutdown_connections 2024-04-26T22:27:19.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 >> 172.21.15.62:0/2025845766 conn(0x7f2e600d0d50 msgr2=0x7f2e600cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 shutdown_connections 2024-04-26T22:27:19.545 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7f2e66d2b640 1 -- 172.21.15.62:0/2025845766 wait complete. 2024-04-26T22:27:19.546 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.491+0000 7fce7dffb640 1 -- 172.21.15.62:0/650450095 <== osd.2 v2:172.21.15.62:6800/1123744749 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce68005c90 con 0x7fce68001650 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.497+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] conn(0x7fce68001650 msgr2=0x7fce68003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.497+0000 7fce737fe640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.62:6800/1123744749,v1:172.21.15.62:6801/1123744749] conn(0x7fce68001650 0x7fce68003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce6c06c9c0 msgr2=0x7fce6c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce6c06c9c0 0x7fce6c06ee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce78034000 tx=0x7fce7802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 msgr2=0x7fce880aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 --2- 172.21.15.62:0/650450095 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce88161140 0x7fce880aee10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fce80001fb0 tx=0x7fce8000a430 comp rx=0 tx=0).stop 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce8dd94640 1 -- 172.21.15.62:0/650450095 reap_dead start 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 shutdown_connections 2024-04-26T22:27:19.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.500+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 >> 172.21.15.62:0/650450095 conn(0x7fce880b2b80 msgr2=0x7fce88072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.501+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 shutdown_connections 2024-04-26T22:27:19.548 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.501+0000 7fce737fe640 1 -- 172.21.15.62:0/650450095 wait complete. 2024-04-26T22:27:19.548 INFO:teuthology.orchestra.run.smithi062.stdout:55834574857 2024-04-26T22:27:19.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T22:27:19.549 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:19.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T22:27:19.550 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:19.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T22:27:19.551 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:19.552 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:19.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T22:27:19.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T22:27:19.553 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:19.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T22:27:19.554 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:19.555 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T22:27:19.555 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:19.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T22:27:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.763+0000 7f2b043e4640 1 Processor -- start 2024-04-26T22:27:19.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.763+0000 7f2b043e4640 1 -- start start 2024-04-26T22:27:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.763+0000 7f7381532640 1 Processor -- start 2024-04-26T22:27:19.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.763+0000 7f7381532640 1 -- start start 2024-04-26T22:27:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.764+0000 7f2b043e4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 0x7f2afc073400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.764+0000 7f2b043e4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2afc0b49f0 con 0x7f2afc0d1770 2024-04-26T22:27:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.764+0000 7f2b043e4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2afc0d0600 con 0x7f2afc0b50e0 2024-04-26T22:27:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.764+0000 7f2b043e4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc0b3c30 con 0x7f2afc072f80 2024-04-26T22:27:19.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f7381532640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f737c1648f0 0x7f737c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f7381532640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f737c0ccb80 0x7f737c167220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f7381532640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f737c0cd090 con 0x7f737c06c240 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f7381532640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f737c163bb0 con 0x7f737c1648f0 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f7381532640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f737c163e10 con 0x7f737c0ccb80 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f2b02981640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f2afc0b50e0 0x7f2afc0b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:52980/0 (socket says 172.21.15.62:52980) 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f2b02981640 1 -- 172.21.15.62:0/3356621173 learned_addr learned my addr 172.21.15.62:0/3356621173 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f2b0197f640 1 --2- 172.21.15.62:0/3356621173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 0x7f2afc073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f737a7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f737c1648f0 0x7f737c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f737affd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f737c06c240 0x7f737c06c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56420/0 (socket says 172.21.15.62:56420) 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f737affd640 1 -- 172.21.15.62:0/2039213679 learned_addr learned my addr 172.21.15.62:0/2039213679 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f737a7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f737c1648f0 0x7f737c0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35506/0 (socket says 172.21.15.62:35506) 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.765+0000 7f737b7fe640 1 --2- 172.21.15.62:0/2039213679 >> v2:172.21.15.174:3300/0 conn(0x7f737c0ccb80 0x7f737c167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f2b0197f640 1 -- 172.21.15.62:0/3356621173 >> v1:172.21.15.174:6789/0 conn(0x7f2afc0b50e0 legacy=0x7f2afc0b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f2b0197f640 1 -- 172.21.15.62:0/3356621173 >> v1:172.21.15.62:6789/0 conn(0x7f2afc0d1770 legacy=0x7f2afc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f2b0197f640 1 -- 172.21.15.62:0/3356621173 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc0b3f10 con 0x7f2afc072f80 2024-04-26T22:27:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f8c75f97640 1 Processor -- start 2024-04-26T22:27:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f8c75f97640 1 -- start start 2024-04-26T22:27:19.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f84ece31640 1 Processor -- start 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7fd175340640 1 Processor -- start 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7fd175340640 1 -- start start 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f737a7fc640 1 -- 172.21.15.62:0/2039213679 >> v2:172.21.15.174:3300/0 conn(0x7f737c0ccb80 msgr2=0x7f737c167220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f737a7fc640 1 --2- 172.21.15.62:0/2039213679 >> v2:172.21.15.174:3300/0 conn(0x7f737c0ccb80 0x7f737c167220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f737a7fc640 1 -- 172.21.15.62:0/2039213679 >> v1:172.21.15.62:6789/0 conn(0x7f737c06c240 legacy=0x7f737c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f737a7fc640 1 -- 172.21.15.62:0/2039213679 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c1640f0 con 0x7f737c1648f0 2024-04-26T22:27:19.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f2b0197f640 1 --2- 172.21.15.62:0/3356621173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 0x7f2afc073400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097d0 tx=0x7f2aec02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ea4660fdbc20e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f737a7fc640 1 --2- 172.21.15.62:0/2039213679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c1648f0 0x7f737c0cc640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f737c072b10 tx=0x7f736c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c963a27032f12d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f8c75f97640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c7006c240 0x7f8c7006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f8c75f97640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 0x7f8c701676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7fec668d1640 1 Processor -- start 2024-04-26T22:27:19.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.766+0000 7f84ece31640 1 -- start start 2024-04-26T22:27:19.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f8c75f97640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7006cb60 con 0x7f8c701648f0 2024-04-26T22:27:19.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f8c75f97640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8c70163bb0 con 0x7f8c7006c240 2024-04-26T22:27:19.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7fec668d1640 1 -- start start 2024-04-26T22:27:19.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f2b0117e640 1 -- 172.21.15.62:0/3356621173 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec037070 con 0x7f2afc072f80 2024-04-26T22:27:19.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd175340640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd168048100 0x7fd168006f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.767+0000 7f8c75f97640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c700cd090 con 0x7f8c70167220 2024-04-26T22:27:19.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f8c6effd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8c701648f0 0x7f8c700cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56426/0 (socket says 172.21.15.62:56426) 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f8c6effd640 1 -- 172.21.15.62:0/3411511886 learned_addr learned my addr 172.21.15.62:0/3411511886 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84ece31640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e004fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84ece31640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 0x7f84e0006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84ece31640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e0054c10 con 0x7f84e004b3c0 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84ece31640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f84e00062e0 con 0x7f84e0006a50 2024-04-26T22:27:19.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84ece31640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f84e0050120 con 0x7f84e0034020 2024-04-26T22:27:19.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f2b0117e640 1 -- 172.21.15.62:0/3356621173 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec032440 con 0x7f2afc072f80 2024-04-26T22:27:19.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd175340640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168054560 0x7fd16804f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f7379ffb640 1 -- 172.21.15.62:0/2039213679 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c037070 con 0x7f737c1648f0 2024-04-26T22:27:19.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd175340640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd168005ec0 con 0x7fd168007460 2024-04-26T22:27:19.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd175340640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd1680e7b90 con 0x7fd168048100 2024-04-26T22:27:19.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84e77fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f84e004b3c0 0x7f84e004f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56440/0 (socket says 172.21.15.62:56440) 2024-04-26T22:27:19.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84e6ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e004fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd175340640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd168006a90 con 0x7fd168054560 2024-04-26T22:27:19.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f84e77fe640 1 -- 172.21.15.62:0/2001973153 learned_addr learned my addr 172.21.15.62:0/2001973153 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f7379ffb640 1 -- 172.21.15.62:0/2039213679 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f736c032440 con 0x7f737c1648f0 2024-04-26T22:27:19.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f7379ffb640 1 -- 172.21.15.62:0/2039213679 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736c032810 con 0x7f737c1648f0 2024-04-26T22:27:19.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd16effd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd168048100 0x7fd168006f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd16effd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd168048100 0x7fd168006f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59286/0 (socket says 172.21.15.62:59286) 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/3411511886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 0x7f8c701676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7f8c6f7fe640 1 --2- 172.21.15.62:0/3411511886 >> v2:172.21.15.62:3300/0 conn(0x7f8c7006c240 0x7f8c7006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f8c6ffff640 1 -- 172.21.15.62:0/3411511886 >> v2:172.21.15.62:3300/0 conn(0x7f8c7006c240 msgr2=0x7f8c7006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/3411511886 >> v2:172.21.15.62:3300/0 conn(0x7f8c7006c240 0x7f8c7006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f8c6ffff640 1 -- 172.21.15.62:0/3411511886 >> v1:172.21.15.62:6789/0 conn(0x7f8c701648f0 legacy=0x7f8c700cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f8c6ffff640 1 -- 172.21.15.62:0/3411511886 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70163e10 con 0x7f8c70167220 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec668d1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 0x7fec60070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec668d1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec668d1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec60073f40 con 0x7fec600ce8e0 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec668d1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fec60073d20 con 0x7fec60145ac0 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec668d1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec6006fc50 con 0x7fec60073890 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fcf89e31640 1 Processor -- start 2024-04-26T22:27:19.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 -- 172.21.15.62:0/3356621173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 msgr2=0x7f2afc073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 --2- 172.21.15.62:0/3356621173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 0x7f2afc073400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097d0 tx=0x7f2aec02a2a0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 -- 172.21.15.62:0/3356621173 shutdown_connections 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 --2- 172.21.15.62:0/3356621173 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc072f80 0x7f2afc073400 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097d0 tx=0x7f2aec02a2a0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 -- 172.21.15.62:0/3356621173 >> 172.21.15.62:0/3356621173 conn(0x7f2afc0d0c80 msgr2=0x7f2afc0d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec64e6e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec64e6e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56048/0 (socket says 172.21.15.62:56048) 2024-04-26T22:27:19.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fec64e6e640 1 -- 172.21.15.62:0/283987975 learned_addr learned my addr 172.21.15.62:0/283987975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e67fc640 1 -- 172.21.15.62:0/2001973153 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1568202222 0 0) 0x7f84e0054c10 con 0x7f84e004b3c0 2024-04-26T22:27:19.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e67fc640 1 -- 172.21.15.62:0/2001973153 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d8003520 con 0x7f84e004b3c0 2024-04-26T22:27:19.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e6ffd640 1 -- 172.21.15.62:0/2001973153 >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 msgr2=0x7f84e0006ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd23e066640 1 Processor -- start 2024-04-26T22:27:19.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fcf89e31640 1 -- start start 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 -- 172.21.15.62:0/3356621173 shutdown_connections 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e7fff640 1 --2- 172.21.15.62:0/2001973153 >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 0x7f84e0006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.768+0000 7fd16effd640 1 -- 172.21.15.62:0/1199182409 learned_addr learned my addr 172.21.15.62:0/1199182409 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd16effd640 1 -- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168054560 msgr2=0x7fd16804f930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd16effd640 1 --2- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168054560 0x7fd16804f930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd16effd640 1 -- 172.21.15.62:0/1199182409 >> v1:172.21.15.174:6789/0 conn(0x7fd168007460 legacy=0x7fd168053e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd16effd640 1 -- 172.21.15.62:0/1199182409 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd168053280 con 0x7fd168048100 2024-04-26T22:27:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fd16effd640 1 --2- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168048100 0x7fd168006f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd164009870 tx=0x7fd16402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd98ce08ff9dffdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e6ffd640 1 --2- 172.21.15.62:0/2001973153 >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 0x7f84e0006ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e6ffd640 1 -- 172.21.15.62:0/2001973153 >> v1:172.21.15.62:6789/0 conn(0x7f84e004b3c0 legacy=0x7f84e004f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f84e6ffd640 1 -- 172.21.15.62:0/2001973153 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e0053a50 con 0x7f84e0034020 2024-04-26T22:27:19.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f7381532640 1 -- 172.21.15.62:0/2039213679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c1648f0 msgr2=0x7f737c0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f7381532640 1 --2- 172.21.15.62:0/2039213679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c1648f0 0x7f737c0cc640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f737c072b10 tx=0x7f736c02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fd16dffb640 1 -- 172.21.15.62:0/1199182409 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd168048100 2024-04-26T22:27:19.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f84e7fff640 1 --2- 172.21.15.62:0/2001973153 >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 0x7f84e0006ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fd16dffb640 1 -- 172.21.15.62:0/1199182409 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164032440 con 0x7fd168048100 2024-04-26T22:27:19.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fd16dffb640 1 -- 172.21.15.62:0/1199182409 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164032810 con 0x7fd168048100 2024-04-26T22:27:19.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fec5ffff640 1 --2- 172.21.15.62:0/283987975 >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 0x7fec60070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89e31640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcf840b5b50 0x7fcf840cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fec64e6e640 1 -- 172.21.15.62:0/283987975 >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 msgr2=0x7fec60070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fec64e6e640 1 --2- 172.21.15.62:0/283987975 >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 0x7fec60070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fec64e6e640 1 -- 172.21.15.62:0/283987975 >> v1:172.21.15.62:6789/0 conn(0x7fec600ce8e0 legacy=0x7fec60073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7fec64e6e640 1 -- 172.21.15.62:0/283987975 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6006ff30 con 0x7fec60073890 2024-04-26T22:27:19.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fec5ffff640 1 --2- 172.21.15.62:0/283987975 >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 0x7fec60070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f84e6ffd640 1 --2- 172.21.15.62:0/2001973153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e004fbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097a0 tx=0x7f84dc02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f77c22d00fb6501e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/3411511886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 0x7f8c701676b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c640097a0 tx=0x7f8c6402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ec4d7e83f7cfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f8c6e7fc640 1 -- 172.21.15.62:0/3411511886 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64037070 con 0x7f8c70167220 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.770+0000 7f8c6e7fc640 1 -- 172.21.15.62:0/3411511886 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6402ae40 con 0x7f8c70167220 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7fd23e066640 1 -- start start 2024-04-26T22:27:19.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 0x7fd23004fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd2300df420 0x7fd2300382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 0x7fd230038c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300068a0 con 0x7fd23004fa90 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300503b0 con 0x7fd2300df420 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23e066640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd230006af0 con 0x7fd230038800 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f84e67fc640 1 -- 172.21.15.62:0/2001973153 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc037070 con 0x7f84e0034020 2024-04-26T22:27:19.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d064640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 0x7fd23004fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d064640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 0x7fd23004fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35548/0 (socket says 172.21.15.62:35548) 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d064640 1 -- 172.21.15.62:0/3146486558 learned_addr learned my addr 172.21.15.62:0/3146486558 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23c863640 1 --2- 172.21.15.62:0/3146486558 >> v2:172.21.15.174:3300/0 conn(0x7fd2300df420 0x7fd2300382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 --2- 172.21.15.62:0/3146486558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 0x7fd230038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89e31640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcf840cdbf0 0x7fcf8415b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.769+0000 7f2b043e4640 1 -- 172.21.15.62:0/3356621173 wait complete. 2024-04-26T22:27:19.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89e31640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf840ce000 con 0x7fcf8415b490 2024-04-26T22:27:19.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f2b043e4640 1 Processor -- start 2024-04-26T22:27:19.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f84e67fc640 1 -- 172.21.15.62:0/2001973153 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc02ab40 con 0x7f84e0034020 2024-04-26T22:27:19.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf88e2f640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fcf8415b490 0x7fcf840d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:52994/0 (socket says 172.21.15.62:52994) 2024-04-26T22:27:19.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f84e67fc640 1 -- 172.21.15.62:0/2001973153 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8050f10 con 0x7f84e0034020 2024-04-26T22:27:19.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf88e2f640 1 -- 172.21.15.62:0/1544217690 learned_addr learned my addr 172.21.15.62:0/1544217690 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:19.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8415fe70 con 0x7fcf840b5b50 2024-04-26T22:27:19.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8415f0b0 con 0x7fcf840cdbf0 2024-04-26T22:27:19.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf89630640 1 --2- 172.21.15.62:0/1544217690 >> v2:172.21.15.174:3300/0 conn(0x7fcf840cdbf0 0x7fcf8415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fcf83fff640 1 --2- 172.21.15.62:0/1544217690 >> v2:172.21.15.62:3300/0 conn(0x7fcf840b5b50 0x7fcf840cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd175340640 1 -- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168048100 msgr2=0x7fd168006f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f8c75f97640 1 -- 172.21.15.62:0/3411511886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 msgr2=0x7f8c701676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f8c75f97640 1 --2- 172.21.15.62:0/3411511886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 0x7f8c701676b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c640097a0 tx=0x7f8c6402a130 comp rx=0 tx=0).stop 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 --2- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168048100 0x7fd168006f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd164009870 tx=0x7fd16402a2c0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 -- 172.21.15.62:0/3146486558 >> v2:172.21.15.174:3300/0 conn(0x7fd2300df420 msgr2=0x7fd2300382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 --2- 172.21.15.62:0/3146486558 >> v2:172.21.15.174:3300/0 conn(0x7fd2300df420 0x7fd2300382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 -- 172.21.15.62:0/3146486558 >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 msgr2=0x7fd23004fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 --2- 172.21.15.62:0/3146486558 >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 0x7fd23004fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7fd23d865640 1 -- 172.21.15.62:0/3146486558 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd230006dd0 con 0x7fd230038800 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 -- 172.21.15.62:0/2039213679 shutdown_connections 2024-04-26T22:27:19.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 --2- 172.21.15.62:0/2039213679 >> v2:172.21.15.174:3300/0 conn(0x7f737c0ccb80 0x7f737c167220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 --2- 172.21.15.62:0/2039213679 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c1648f0 0x7f737c0cc640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 -- 172.21.15.62:0/2039213679 >> 172.21.15.62:0/2039213679 conn(0x7f737c0ce7d0 msgr2=0x7f737c0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f8c75f97640 1 -- 172.21.15.62:0/3411511886 shutdown_connections 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f8c75f97640 1 --2- 172.21.15.62:0/3411511886 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c70167220 0x7f8c701676b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f8c75f97640 1 --2- 172.21.15.62:0/3411511886 >> v2:172.21.15.62:3300/0 conn(0x7f8c7006c240 0x7f8c7006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f8c75f97640 1 -- 172.21.15.62:0/3411511886 >> 172.21.15.62:0/3411511886 conn(0x7f8c700ce7d0 msgr2=0x7f8c700cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fec64e6e640 1 --2- 172.21.15.62:0/283987975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec480098a0 tx=0x7fec4802a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36a58cf002fa48ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.771+0000 7f2b043e4640 1 -- start start 2024-04-26T22:27:19.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 -- 172.21.15.62:0/1199182409 shutdown_connections 2024-04-26T22:27:19.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 --2- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168054560 0x7fd16804f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 -- 172.21.15.62:0/2039213679 shutdown_connections 2024-04-26T22:27:19.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 --2- 172.21.15.62:0/1199182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168048100 0x7fd168006f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 -- 172.21.15.62:0/1199182409 >> 172.21.15.62:0/1199182409 conn(0x7fd168009fe0 msgr2=0x7fd16800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fec5effd640 1 -- 172.21.15.62:0/283987975 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48037070 con 0x7fec60073890 2024-04-26T22:27:19.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 -- 172.21.15.62:0/1199182409 shutdown_connections 2024-04-26T22:27:19.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fd175340640 1 -- 172.21.15.62:0/1199182409 wait complete. 2024-04-26T22:27:19.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fec5effd640 1 -- 172.21.15.62:0/283987975 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec48032440 con 0x7fec60073890 2024-04-26T22:27:19.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 -- 172.21.15.62:0/2001973153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 msgr2=0x7f84e004fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 0x7f2afc15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2afc169d60 0x7f2afc153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc06f360 con 0x7f2afc0b50e0 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0b44d0 con 0x7f2afc169d60 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f2b043e4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc06f640 con 0x7f2afc0d1770 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fcf83fff640 1 -- 172.21.15.62:0/1544217690 >> v2:172.21.15.174:3300/0 conn(0x7fcf840cdbf0 msgr2=0x7fcf8415b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fcf837fe640 1 -- 172.21.15.62:0/1544217690 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962325282 0 0) 0x7fcf6c001110 con 0x7fcf8415b490 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fcf83fff640 1 --2- 172.21.15.62:0/1544217690 >> v2:172.21.15.174:3300/0 conn(0x7fcf840cdbf0 0x7fcf8415b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fcf83fff640 1 -- 172.21.15.62:0/1544217690 >> v1:172.21.15.174:6789/0 conn(0x7fcf8415b490 legacy=0x7fcf840d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fcf83fff640 1 -- 172.21.15.62:0/1544217690 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8415f390 con 0x7fcf840b5b50 2024-04-26T22:27:19.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f7381532640 1 -- 172.21.15.62:0/2039213679 wait complete. 2024-04-26T22:27:19.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7fec5effd640 1 -- 172.21.15.62:0/283987975 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48032810 con 0x7fec60073890 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b02981640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2afc169d60 0x7f2afc153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56078/0 (socket says 172.21.15.62:56078) 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 --2- 172.21.15.62:0/2001973153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e004fbe0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097a0 tx=0x7f84dc02a0b0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 -- 172.21.15.62:0/2001973153 shutdown_connections 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 --2- 172.21.15.62:0/2001973153 >> v2:172.21.15.62:3300/0 conn(0x7f84e0006a50 0x7f84e0006ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 --2- 172.21.15.62:0/2001973153 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e004fbe0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 -- 172.21.15.62:0/2001973153 >> 172.21.15.62:0/2001973153 conn(0x7f84e000a400 msgr2=0x7f84e000a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f84ece31640 1 -- 172.21.15.62:0/2001973153 shutdown_connections 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f84ece31640 1 -- 172.21.15.62:0/2001973153 wait complete. 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 -- 172.21.15.62:0/1812629744 learned_addr learned my addr 172.21.15.62:0/1812629744 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b02180640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 0x7f2afc15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.772+0000 7f8c75f97640 1 -- 172.21.15.62:0/3411511886 shutdown_connections 2024-04-26T22:27:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fd23d865640 1 --2- 172.21.15.62:0/3146486558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 0x7fd230038c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd230007a60 tx=0x7fd23402e520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e28b7c6e5d01fc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 -- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2afc169d60 msgr2=0x7f2afc153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2afc169d60 0x7f2afc153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 -- 172.21.15.62:0/1812629744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 msgr2=0x7f2afc15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fcf83fff640 1 --2- 172.21.15.62:0/1544217690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf840cd6b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf840d1d80 tx=0x7fcf7c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c44210cc8b2c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fcf837fe640 1 -- 172.21.15.62:0/1544217690 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c040070 con 0x7fcf840b5b50 2024-04-26T22:27:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 0x7f2afc15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f8c75f97640 1 -- 172.21.15.62:0/3411511886 wait complete. 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 -- 172.21.15.62:0/283987975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 msgr2=0x7fec600ced70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 --2- 172.21.15.62:0/283987975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec480098a0 tx=0x7fec4802a1f0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 -- 172.21.15.62:0/283987975 shutdown_connections 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 --2- 172.21.15.62:0/283987975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec60073890 0x7fec600ced70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 --2- 172.21.15.62:0/283987975 >> v2:172.21.15.174:3300/0 conn(0x7fec60145ac0 0x7fec60070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 -- 172.21.15.62:0/283987975 >> 172.21.15.62:0/283987975 conn(0x7fec600b3be0 msgr2=0x7fec600b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7fec668d1640 1 -- 172.21.15.62:0/283987975 shutdown_connections 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fec668d1640 1 -- 172.21.15.62:0/283987975 wait complete. 2024-04-26T22:27:19.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fcf837fe640 1 -- 172.21.15.62:0/1544217690 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c033b70 con 0x7fcf840b5b50 2024-04-26T22:27:19.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.773+0000 7f2b0197f640 1 -- 172.21.15.62:0/1812629744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc142cb0 con 0x7f2afc0d1770 2024-04-26T22:27:19.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7f2b0197f640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097d0 tx=0x7f2aec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7f2b02180640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 0x7f2afc15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:19.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec037050 con 0x7f2afc0d1770 2024-04-26T22:27:19.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fd175340640 1 Processor -- start 2024-04-26T22:27:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fd22ffff640 1 -- 172.21.15.62:0/3146486558 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23403a070 con 0x7fd230038800 2024-04-26T22:27:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1690a0 con 0x7f2afc0d1770 2024-04-26T22:27:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec043070 con 0x7f2afc0d1770 2024-04-26T22:27:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fd22ffff640 1 -- 172.21.15.62:0/3146486558 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234033710 con 0x7fd230038800 2024-04-26T22:27:19.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7fd22ffff640 1 -- 172.21.15.62:0/3146486558 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234033ae0 con 0x7fd230038800 2024-04-26T22:27:19.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc15b220 con 0x7f2afc0d1770 2024-04-26T22:27:19.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f7381532640 1 Processor -- start 2024-04-26T22:27:19.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f7381532640 1 -- start start 2024-04-26T22:27:19.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f8c75f97640 1 Processor -- start 2024-04-26T22:27:19.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 -- 172.21.15.62:0/3146486558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 msgr2=0x7fd230038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 --2- 172.21.15.62:0/3146486558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 0x7fd230038c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd230007a60 tx=0x7fd23402e520 comp rx=0 tx=0).stop 2024-04-26T22:27:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 msgr2=0x7fcf840cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7fcf89e31640 1 --2- 172.21.15.62:0/1544217690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf840cd6b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf840d1d80 tx=0x7fcf7c032e20 comp rx=0 tx=0).stop 2024-04-26T22:27:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7fd175340640 1 -- start start 2024-04-26T22:27:19.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7fec668d1640 1 Processor -- start 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7fec668d1640 1 -- start start 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 -- 172.21.15.62:0/3146486558 shutdown_connections 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 --2- 172.21.15.62:0/3146486558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd230038800 0x7fd230038c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 --2- 172.21.15.62:0/3146486558 >> v2:172.21.15.174:3300/0 conn(0x7fd2300df420 0x7fd2300382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 --2- 172.21.15.62:0/3146486558 >> v2:172.21.15.62:3300/0 conn(0x7fd23004fa90 0x7fd23004fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 -- 172.21.15.62:0/3146486558 >> 172.21.15.62:0/3146486558 conn(0x7fd230009fb0 msgr2=0x7fd23000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f7381532640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec030d00 con 0x7f2afc0d1770 2024-04-26T22:27:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 -- 172.21.15.62:0/3146486558 shutdown_connections 2024-04-26T22:27:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f7381532640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f737c0ccb80 0x7f737c0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f737affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7fd23e066640 1 -- 172.21.15.62:0/3146486558 wait complete. 2024-04-26T22:27:19.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f7381532640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 0x7f737c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4005290 con 0x7f2afc0d1770 2024-04-26T22:27:19.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f7381532640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f737c073cf0 con 0x7f737c06c240 2024-04-26T22:27:19.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f7381532640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f737c0cd370 con 0x7f737c0ccb80 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f7381532640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f737c073e70 con 0x7f737c1648f0 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec600ce8e0 0x7fec600cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec60145ac0 0x7fec60165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec600b5be0 con 0x7fec60073890 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec600b5a60 con 0x7fec60145ac0 2024-04-26T22:27:19.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec668d1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec600b5d60 con 0x7fec600ce8e0 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35580/0 (socket says 172.21.15.62:35580) 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 -- 172.21.15.62:0/3031564295 learned_addr learned my addr 172.21.15.62:0/3031564295 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737b7fe640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 0x7f737c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737a7fc640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f737c0ccb80 0x7f737c0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 0x7fd1680e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f8c75f97640 1 -- start start 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 0x7fd1680e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168054560 0x7fd1680e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd168006720 con 0x7fd168007460 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd168006440 con 0x7fd168054560 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd175340640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd16804fe70 con 0x7fd168048100 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 -- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 msgr2=0x7f737c0b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 0x7f737c0b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 -- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f737c0ccb80 msgr2=0x7f737c0b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f737c0ccb80 0x7f737c0b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 -- 172.21.15.62:0/3031564295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f737c0c95a0 con 0x7f737c06c240 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2aec048460 con 0x7f2afc0d1770 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737b7fe640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 0x7f737c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.774+0000 7f84ece31640 1 Processor -- start 2024-04-26T22:27:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.775+0000 7f84ece31640 1 -- start start 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 0x7f84e00e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e00e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 0x7f84e00e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec64e6e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec60145ac0 0x7fec60165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84e004b7a0 con 0x7f84e0006a50 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0006530 con 0x7f84e004b3c0 2024-04-26T22:27:19.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84ece31640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f84e0054830 con 0x7f84e0034020 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84e7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 0x7f84e00e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84e6ffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e00e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84e7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 0x7f84e00e1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59324/0 (socket says 172.21.15.62:59324) 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.776+0000 7f84e7fff640 1 -- 172.21.15.62:0/3218730662 learned_addr learned my addr 172.21.15.62:0/3218730662 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e77fe640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 0x7f84e00e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 -- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 msgr2=0x7f84e00e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 0x7f84e00e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 -- 172.21.15.62:0/3218730662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 msgr2=0x7f84e00e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 0x7f84e00e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec5ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35594/0 (socket says 172.21.15.62:35594) 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fec5ffff640 1 -- 172.21.15.62:0/3851161508 learned_addr learned my addr 172.21.15.62:0/3851161508 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 -- 172.21.15.62:0/3218730662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e00e81b0 con 0x7f84e0034020 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e77fe640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 0x7f84e00e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f84e6ffd640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e00e7580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031000 tx=0x7f84dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7fd16effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 0x7fd1680e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 0x7fd1680e0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35596/0 (socket says 172.21.15.62:35596) 2024-04-26T22:27:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16effd640 1 -- 172.21.15.62:0/2172932350 learned_addr learned my addr 172.21.15.62:0/2172932350 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f737affd640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f737400cbc0 tx=0x7f737400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5f7fe640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec600ce8e0 0x7fec600cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 0x7f8c701fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 0x7f8c700c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 0x7f8c700b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c700cd370 con 0x7f8c701648f0 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c701643d0 con 0x7f8c70167220 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.777+0000 7f8c75f97640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c700ccd50 con 0x7f8c7006c240 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f2af2ffd640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 0x7f2ae006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2aec0ac650 con 0x7f2afc0d1770 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 0x7f8c701fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f8c6ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 0x7f8c700b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 shutdown_connections 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 --2- 172.21.15.62:0/1544217690 >> v2:172.21.15.174:3300/0 conn(0x7fcf840cdbf0 0x7fcf8415b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 --2- 172.21.15.62:0/1544217690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf840cd6b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 >> 172.21.15.62:0/1544217690 conn(0x7fcf840d0ca0 msgr2=0x7fcf840d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16f7fe640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168054560 0x7fd1680e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 0x7fd1680e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5ffff640 1 -- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec600ce8e0 msgr2=0x7fec600cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f8c6f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 0x7f8c701fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56136/0 (socket says 172.21.15.62:56136) 2024-04-26T22:27:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f8c6f7fe640 1 -- 172.21.15.62:0/1558969192 learned_addr learned my addr 172.21.15.62:0/1558969192 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5ffff640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec600ce8e0 0x7fec600cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5ffff640 1 -- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec60145ac0 msgr2=0x7fec60165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f2b02180640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 0x7f2ae006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5ffff640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec60145ac0 0x7fec60165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.822 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 shutdown_connections 2024-04-26T22:27:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7f8c6effd640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 0x7f8c700c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fec5ffff640 1 -- 172.21.15.62:0/3851161508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6015c220 con 0x7fec60073890 2024-04-26T22:27:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd23e066640 1 Processor -- start 2024-04-26T22:27:19.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd23e066640 1 -- start start 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 -- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168054560 msgr2=0x7fd1680e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168054560 0x7fd1680e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 -- 172.21.15.62:0/2172932350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 msgr2=0x7fd1680e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 0x7fd1680e0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fd16e7fc640 1 -- 172.21.15.62:0/2172932350 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1680e21a0 con 0x7fd168048100 2024-04-26T22:27:19.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374017070 con 0x7f737c06c240 2024-04-26T22:27:19.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec5ffff640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ce00 tx=0x7fec5800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.778+0000 7fcf89e31640 1 -- 172.21.15.62:0/1544217690 wait complete. 2024-04-26T22:27:19.825 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737400aca0 con 0x7f737c06c240 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 -- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 msgr2=0x7f8c701fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 0x7f8c701fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 -- 172.21.15.62:0/1558969192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 msgr2=0x7f8c700c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 0x7f8c700c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 -- 172.21.15.62:0/1558969192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c700b4870 con 0x7f8c70167220 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6effd640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 0x7f8c700c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:19.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f2b02180640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 0x7f2ae006ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000f60 tx=0x7f2af8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fd16e7fc640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 0x7fd1680e17c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd17006b160 tx=0x7fd170069ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800f890 con 0x7fec60073890 2024-04-26T22:27:19.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec668d1640 1 -- 172.21.15.62:0/3851161508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec60141f50 con 0x7fec60073890 2024-04-26T22:27:19.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5800fa30 con 0x7fec60073890 2024-04-26T22:27:19.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800fe00 con 0x7fec60073890 2024-04-26T22:27:19.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c6ffff640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 0x7f8c700b6b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c64031000 tx=0x7f8c640046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fec668d1640 1 -- 172.21.15.62:0/3851161508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec60142510 con 0x7fec60073890 2024-04-26T22:27:19.828 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64037040 con 0x7f8c70167220 2024-04-26T22:27:19.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c64043070 con 0x7f8c70167220 2024-04-26T22:27:19.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64030ad0 con 0x7f8c70167220 2024-04-26T22:27:19.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.779+0000 7fd23e066640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd230038800 0x7fd2300dfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23e066640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7374012600 con 0x7f737c06c240 2024-04-26T22:27:19.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23e066640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 0x7fd230049500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f737c0c9880 con 0x7f737c06c240 2024-04-26T22:27:19.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23c863640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23e066640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd230052b40 con 0x7fd230038800 2024-04-26T22:27:19.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c700b4b50 con 0x7f8c70167220 2024-04-26T22:27:19.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fcf89e31640 1 Processor -- start 2024-04-26T22:27:19.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f737c0c9e40 con 0x7f737c06c240 2024-04-26T22:27:19.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fcf89e31640 1 -- start start 2024-04-26T22:27:19.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23c863640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56140/0 (socket says 172.21.15.62:56140) 2024-04-26T22:27:19.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c700b5110 con 0x7f8c70167220 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23c863640 1 -- 172.21.15.62:0/4045596947 learned_addr learned my addr 172.21.15.62:0/4045596947 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23d865640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 0x7fd230049500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd230007390 con 0x7fd2300df420 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd230052cc0 con 0x7fd23004fa90 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.780+0000 7fd23d064640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd230038800 0x7fd2300dfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 -- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 msgr2=0x7fd230049500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 0x7fd230049500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 -- 172.21.15.62:0/4045596947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd230038800 msgr2=0x7fd2300dfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd230038800 0x7fd2300dfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2300e04b0 con 0x7fd23004fa90 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23d865640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 0x7fd230049500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fd23c863640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd2280089e0 tx=0x7fd228008ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fec668d1640 1 -- 172.21.15.62:0/3851161508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec600693e0 con 0x7fec60073890 2024-04-26T22:27:19.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f737c076900 con 0x7f737c06c240 2024-04-26T22:27:19.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fcf89e31640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf841467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.781+0000 7fcf89e31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf89e31640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcf8415b490 0x7fcf8414f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf88e2f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf841467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf89e31640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8406fff0 con 0x7fcf840b5b50 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf89e31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8415f950 con 0x7fcf840cdbf0 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf89e31640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcf84070170 con 0x7fcf8415b490 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:59382/0 (socket says 172.21.15.62:59382) 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 -- 172.21.15.62:0/3464418183 learned_addr learned my addr 172.21.15.62:0/3464418183 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf89630640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcf8415b490 0x7fcf8414f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c70076900 con 0x7f8c70167220 2024-04-26T22:27:19.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 -- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcf8415b490 msgr2=0x7fcf8414f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcf8415b490 0x7fcf8414f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 -- 172.21.15.62:0/3464418183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 msgr2=0x7fcf841467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:19.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec58018400 con 0x7fec60073890 2024-04-26T22:27:19.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf841467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:19.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7fcf83fff640 1 -- 172.21.15.62:0/3464418183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf84152d70 con 0x7fcf840cdbf0 2024-04-26T22:27:19.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f737401a070 con 0x7f737c06c240 2024-04-26T22:27:19.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.782+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c64030c70 con 0x7f8c70167220 2024-04-26T22:27:19.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7fcf83fff640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c033410 tx=0x7fcf7c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f73637fe640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 0x7f736806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7fec5cff9640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f737408b680 con 0x7f737c06c240 2024-04-26T22:27:19.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f8c57fff640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 0x7f8c5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fec5808c720 con 0x7fec60073890 2024-04-26T22:27:19.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f737a7fc640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 0x7f736806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c640ac060 con 0x7f8c70167220 2024-04-26T22:27:19.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7fec5f7fe640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc037040 con 0x7f84e0034020 2024-04-26T22:27:19.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22801a070 con 0x7fd23004fa90 2024-04-26T22:27:19.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd228009d20 con 0x7fd23004fa90 2024-04-26T22:27:19.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc004230 con 0x7f84e0034020 2024-04-26T22:27:19.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc003610 con 0x7f84e0034020 2024-04-26T22:27:19.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f84ece31640 1 -- 172.21.15.62:0/3218730662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e00e8490 con 0x7f84e0034020 2024-04-26T22:27:19.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2300e0760 con 0x7fd23004fa90 2024-04-26T22:27:19.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170072d20 con 0x7fd168048100 2024-04-26T22:27:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2aec07a530 con 0x7f2afc0d1770 2024-04-26T22:27:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f84ece31640 1 -- 172.21.15.62:0/3218730662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e004c2d0 con 0x7f84e0034020 2024-04-26T22:27:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2300e0d20 con 0x7fd23004fa90 2024-04-26T22:27:19.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f737a7fc640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 0x7f736806ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f736c034000 tx=0x7f736c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.783+0000 7f8c6f7fe640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 0x7f8c5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7f8c6f7fe640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 0x7f8c5c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8c680094c0 tx=0x7f8c6800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd175340640 1 -- 172.21.15.62:0/2172932350 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1680e2480 con 0x7fd168048100 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd175340640 1 -- 172.21.15.62:0/2172932350 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1680e2a40 con 0x7fd168048100 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd170074440 con 0x7fd168048100 2024-04-26T22:27:19.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.785+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1700748b0 con 0x7fd168048100 2024-04-26T22:27:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fec5f7fe640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 0x7fec4006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec60145310 tx=0x7fec50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.784+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228015660 con 0x7fd23004fa90 2024-04-26T22:27:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.785+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2300089d0 con 0x7fd23004fa90 2024-04-26T22:27:19.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84dc0037b0 con 0x7f84e0034020 2024-04-26T22:27:19.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7fd175340640 1 -- 172.21.15.62:0/2172932350 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd150005290 con 0x7fd168048100 2024-04-26T22:27:19.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7f84ece31640 1 -- 172.21.15.62:0/3218730662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c8005290 con 0x7f84e0034020 2024-04-26T22:27:19.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7f84d3fff640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd17006f030 con 0x7fd168048100 2024-04-26T22:27:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f84dc0ce9d0 con 0x7f84e0034020 2024-04-26T22:27:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.786+0000 7f84e77fe640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd22801d070 con 0x7fd23004fa90 2024-04-26T22:27:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7f84e77fe640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 0x7f84cc06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f84e8063ef0 tx=0x7f84e80660f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd1577fe640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 0x7fd15c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd22dffb640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 0x7fd21c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd1701084e0 con 0x7fd168048100 2024-04-26T22:27:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd22808b600 con 0x7fd23004fa90 2024-04-26T22:27:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd16effd640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 0x7fd15c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.787+0000 7fd23d064640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 0x7fd21c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fd23d064640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 0x7fd21c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd23807bb70 tx=0x7fd23806a160 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fd16effd640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 0x7fd15c06ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd164009280 tx=0x7fd16402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c04b020 con 0x7fcf840cdbf0 2024-04-26T22:27:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c046750 con 0x7fcf840cdbf0 2024-04-26T22:27:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c040040 con 0x7fcf840cdbf0 2024-04-26T22:27:19.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8414dfd0 con 0x7fcf840cdbf0 2024-04-26T22:27:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8414e590 con 0x7fcf840cdbf0 2024-04-26T22:27:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.788+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c6407a3f0 con 0x7f8c70167220 2024-04-26T22:27:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf84069030 con 0x7fcf840cdbf0 2024-04-26T22:27:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf7c04e020 con 0x7fcf840cdbf0 2024-04-26T22:27:19.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec58014030 con 0x7fec60073890 2024-04-26T22:27:19.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fcf817fa640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 0x7fcf6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:19.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c0b53d0 con 0x7fcf840cdbf0 2024-04-26T22:27:19.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.790+0000 7fcf88e2f640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 0x7fcf6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:19.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.791+0000 7fcf88e2f640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 0x7fcf6406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.792+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd170090030 con 0x7fd168048100 2024-04-26T22:27:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.795+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73740595d0 con 0x7f737c06c240 2024-04-26T22:27:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.795+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf7c055030 con 0x7fcf840cdbf0 2024-04-26T22:27:19.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.796+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84dc09cd60 con 0x7f84e0034020 2024-04-26T22:27:19.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:19.801+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2280593d0 con 0x7fd23004fa90 2024-04-26T22:27:20.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.221+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2ae4005580 con 0x7f2afc0d1770 2024-04-26T22:27:20.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.222+0000 7f2af2ffd640 1 -- 172.21.15.62:0/1812629744 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2aec07e830 con 0x7f2afc0d1770 2024-04-26T22:27:20.222 INFO:teuthology.orchestra.run.smithi062.stdout:55834574857 2024-04-26T22:27:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.223+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8c700cf4f0 con 0x7f8c70167220 2024-04-26T22:27:20.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.223+0000 7f8c57fff640 1 -- 172.21.15.62:0/1558969192 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8c64079cf0 con 0x7f8c70167220 2024-04-26T22:27:20.223 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:20.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.224+0000 7fd175340640 1 -- 172.21.15.62:0/2172932350 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd150005850 con 0x7fd168048100 2024-04-26T22:27:20.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.225+0000 7fd1577fe640 1 -- 172.21.15.62:0/2172932350 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1700d63e0 con 0x7fd168048100 2024-04-26T22:27:20.225 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:20.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.225+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 msgr2=0x7f2ae006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 0x7f2ae006ee80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000f60 tx=0x7f2af8009040 comp rx=0 tx=0).stop 2024-04-26T22:27:20.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7fec668d1640 1 -- 172.21.15.62:0/3851161508 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fec6006f660 con 0x7fec60073890 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 msgr2=0x7f2afc1421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0097d0 tx=0x7f2aec0046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 shutdown_connections 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2ae006c9c0 0x7f2ae006ee80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2afc169d60 0x7f2afc153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2afc0d1770 0x7f2afc1421d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7fec5cff9640 1 -- 172.21.15.62:0/3851161508 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fec5800ae20 con 0x7fec60073890 2024-04-26T22:27:20.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.226+0000 7f2b043e4640 1 --2- 172.21.15.62:0/1812629744 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2afc0b50e0 0x7f2afc15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.228 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:20.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 >> 172.21.15.62:0/1812629744 conn(0x7f2afc0d0c80 msgr2=0x7f2afc06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 msgr2=0x7f8c5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 shutdown_connections 2024-04-26T22:27:20.230 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f2b043e4640 1 -- 172.21.15.62:0/1812629744 wait complete. 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 0x7f8c5c06edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8c680094c0 tx=0x7f8c6800d040 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 msgr2=0x7f8c700b6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 0x7f8c700b6b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c64031000 tx=0x7f8c640046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 shutdown_connections 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c5c06c8f0 0x7f8c5c06edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c70167220 0x7f8c700b6b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c701648f0 0x7f8c700c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 --2- 172.21.15.62:0/1558969192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7006c240 0x7f8c701fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.227+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 >> 172.21.15.62:0/1558969192 conn(0x7f8c700ce7d0 msgr2=0x7f8c700cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 shutdown_connections 2024-04-26T22:27:20.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7f8c75f97640 1 -- 172.21.15.62:0/1558969192 wait complete. 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 msgr2=0x7fd15c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 0x7fd15c06ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd164009280 tx=0x7fd16402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 msgr2=0x7fd1680e17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 0x7fd1680e17c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd17006b160 tx=0x7fd170069ec0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 shutdown_connections 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd15c06c940 0x7fd15c06ee00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd168054560 0x7fd1680e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd168048100 0x7fd1680e17c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 --2- 172.21.15.62:0/2172932350 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd168007460 0x7fd1680e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.228+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 >> 172.21.15.62:0/2172932350 conn(0x7fd168009fe0 msgr2=0x7fd168036e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.229+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 shutdown_connections 2024-04-26T22:27:20.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.229+0000 7fd1557fa640 1 -- 172.21.15.62:0/2172932350 wait complete. 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.230+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 msgr2=0x7fec4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.230+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 0x7fec4006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec60145310 tx=0x7fec50009040 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.230+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 msgr2=0x7fec6015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.230+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ce00 tx=0x7fec5800a650 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 shutdown_connections 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec4006c8f0 0x7fec4006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec60145ac0 0x7fec60165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec600ce8e0 0x7fec600cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 --2- 172.21.15.62:0/3851161508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec60073890 0x7fec6015bce0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 >> 172.21.15.62:0/3851161508 conn(0x7fec600b3be0 msgr2=0x7fec60140520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.231+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 shutdown_connections 2024-04-26T22:27:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.232+0000 7fec567fc640 1 -- 172.21.15.62:0/3851161508 wait complete. 2024-04-26T22:27:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.234+0000 7f84ece31640 1 -- 172.21.15.62:0/3218730662 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f84c8005580 con 0x7f84e0034020 2024-04-26T22:27:20.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.235+0000 7f84d3fff640 1 -- 172.21.15.62:0/3218730662 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f84dc09c660 con 0x7f84e0034020 2024-04-26T22:27:20.236 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.234+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f737c0cf4f0 con 0x7f737c06c240 2024-04-26T22:27:20.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.235+0000 7f73637fe640 1 -- 172.21.15.62:0/3031564295 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f737405d8d0 con 0x7f737c06c240 2024-04-26T22:27:20.237 INFO:teuthology.orchestra.run.smithi062.stdout:55834574855 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 msgr2=0x7f84cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 0x7f84cc06edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f84e8063ef0 tx=0x7f84e80660f0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 msgr2=0x7f84e00e7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e00e7580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f84dc031000 tx=0x7f84dc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 shutdown_connections 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f84cc06c8f0 0x7f84cc06edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84e004b3c0 0x7f84e00e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84e0034020 0x7f84e00e7580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 --2- 172.21.15.62:0/3218730662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f84e0006a50 0x7f84e00e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 >> 172.21.15.62:0/3218730662 conn(0x7f84e000a400 msgr2=0x7f84e0035340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd230005390 con 0x7fd23004fa90 2024-04-26T22:27:20.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 shutdown_connections 2024-04-26T22:27:20.241 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f84d1ffb640 1 -- 172.21.15.62:0/3218730662 wait complete. 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 msgr2=0x7f736806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 0x7f736806ee00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f736c034000 tx=0x7f736c02e000 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 msgr2=0x7f737c158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.239+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f737400cbc0 tx=0x7f737400a060 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 shutdown_connections 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f736806c940 0x7f736806ee00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f737c1648f0 0x7f737c0b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f737c0ccb80 0x7f737c0b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 --2- 172.21.15.62:0/3031564295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f737c06c240 0x7f737c158350 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 >> 172.21.15.62:0/3031564295 conn(0x7f737c0ce7d0 msgr2=0x7f737c06f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 shutdown_connections 2024-04-26T22:27:20.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7f7381532640 1 -- 172.21.15.62:0/3031564295 wait complete. 2024-04-26T22:27:20.243 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:20.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.240+0000 7fd22dffb640 1 -- 172.21.15.62:0/4045596947 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd22805d6d0 con 0x7fd23004fa90 2024-04-26T22:27:20.244 INFO:teuthology.orchestra.run.smithi062.stdout:55834574856 2024-04-26T22:27:20.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.243+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcf84076890 con 0x7fcf840cdbf0 2024-04-26T22:27:20.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.244+0000 7fcf817fa640 1 -- 172.21.15.62:0/3464418183 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf7c083220 con 0x7fcf840cdbf0 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.244+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 msgr2=0x7fd21c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.244+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 0x7fd21c06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd23807bb70 tx=0x7fd23806a160 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.244+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 msgr2=0x7fd2300dff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.244+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd2280089e0 tx=0x7fd228008ad0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 shutdown_connections 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd21c06c8f0 0x7fd21c06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2300df420 0x7fd230049500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd23004fa90 0x7fd2300dff70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 --2- 172.21.15.62:0/4045596947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd230038800 0x7fd2300dfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 >> 172.21.15.62:0/4045596947 conn(0x7fd230009fb0 msgr2=0x7fd230038db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 shutdown_connections 2024-04-26T22:27:20.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.245+0000 7fd23e066640 1 -- 172.21.15.62:0/4045596947 wait complete. 2024-04-26T22:27:20.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-26T22:27:20.248 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.248+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 msgr2=0x7fcf6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.248+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 0x7fcf6406edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c007b50 comp rx=0 tx=0).stop 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.248+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 msgr2=0x7fcf8414ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.248+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c033410 tx=0x7fcf7c004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 shutdown_connections 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcf6406c8f0 0x7fcf6406edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcf8415b490 0x7fcf8414f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf840cdbf0 0x7fcf8414ecc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 --2- 172.21.15.62:0/3464418183 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf840b5b50 0x7fcf841467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 >> 172.21.15.62:0/3464418183 conn(0x7fcf840d0ca0 msgr2=0x7fcf8414ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.249+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 shutdown_connections 2024-04-26T22:27:20.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.250+0000 7fcf89e31640 1 -- 172.21.15.62:0/3464418183 wait complete. 2024-04-26T22:27:20.251 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-04-26T22:27:20.252 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-26T22:27:20.252 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-04-26T22:27:20.256 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-26T22:27:20.259 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-26T22:27:20.260 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-04-26T22:27:20.263 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2024-04-26T22:27:20.264 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:20.264 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T22:27:20.265 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:27:20.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.406+0000 7f8edaa7e640 1 Processor -- start 2024-04-26T22:27:20.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.406+0000 7f8edaa7e640 1 -- start start 2024-04-26T22:27:20.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8edaa7e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed40d22e0 0x7f8ed40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8edaa7e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ed415f760 0x7f8ed40b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8edaa7e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed40d3de0 con 0x7f8ed415f380 2024-04-26T22:27:20.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8edaa7e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed40d43a0 con 0x7f8ed415f760 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8edaa7e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed40d40c0 con 0x7f8ed40d22e0 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed901b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8ed415f760 0x7f8ed40b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed881a640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8ed415f380 0x7f8ed40d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56462/0 (socket says 172.21.15.62:56462) 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed881a640 1 -- 172.21.15.62:0/260554764 learned_addr learned my addr 172.21.15.62:0/260554764 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed3fff640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed40d22e0 0x7f8ed40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed901b640 1 -- 172.21.15.62:0/260554764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed40d22e0 msgr2=0x7f8ed40b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed901b640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed40d22e0 0x7f8ed40b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed901b640 1 -- 172.21.15.62:0/260554764 >> v1:172.21.15.62:6789/0 conn(0x7f8ed415f380 legacy=0x7f8ed40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed901b640 1 -- 172.21.15.62:0/260554764 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed40b8280 con 0x7f8ed415f760 2024-04-26T22:27:20.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.407+0000 7f8ed37fe640 1 -- 172.21.15.62:0/260554764 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 856532963 0 0) 0x7f8ed40d3de0 con 0x7f8ed415f380 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8ed901b640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed415f760 0x7f8ed40b8fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8cf45515c8dfce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8ed37fe640 1 -- 172.21.15.62:0/260554764 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc007740 con 0x7f8ed415f760 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8ed37fe640 1 -- 172.21.15.62:0/260554764 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc0078e0 con 0x7f8ed415f760 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8ed37fe640 1 -- 172.21.15.62:0/260554764 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc007cb0 con 0x7f8ed415f760 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 -- 172.21.15.62:0/260554764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed415f760 msgr2=0x7f8ed40b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed415f760 0x7f8ed40b8fc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009d90 tx=0x7f8ecc02a6f0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 -- 172.21.15.62:0/260554764 shutdown_connections 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed415f760 0x7f8ed40b8fc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/260554764 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed40d22e0 0x7f8ed40b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 -- 172.21.15.62:0/260554764 >> 172.21.15.62:0/260554764 conn(0x7f8ed409e560 msgr2=0x7f8ed40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 -- 172.21.15.62:0/260554764 shutdown_connections 2024-04-26T22:27:20.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.408+0000 7f8edaa7e640 1 -- 172.21.15.62:0/260554764 wait complete. 2024-04-26T22:27:20.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.410+0000 7f8edaa7e640 1 Processor -- start 2024-04-26T22:27:20.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.410+0000 7f8edaa7e640 1 -- start start 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.410+0000 7f8edaa7e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.410+0000 7f8edaa7e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed415f380 0x7f8ed406d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 0x7f8ed415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed40d1280 con 0x7f8ed40d22e0 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed40d1100 con 0x7f8ed415f380 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed40d5b50 con 0x7f8ed415f760 2024-04-26T22:27:20.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35636/0 (socket says 172.21.15.62:35636) 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 -- 172.21.15.62:0/1291116452 learned_addr learned my addr 172.21.15.62:0/1291116452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed901b640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 0x7f8ed415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed3fff640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed415f380 0x7f8ed406d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 -- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 msgr2=0x7f8ed415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 0x7f8ed415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 -- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed415f380 msgr2=0x7f8ed406d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed415f380 0x7f8ed406d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 -- 172.21.15.62:0/1291116452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed415bc10 con 0x7f8ed40d22e0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed901b640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 0x7f8ed415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed881a640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c500 tx=0x7f8ec800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8018070 con 0x7f8ed40d22e0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec8007580 con 0x7f8ed40d22e0 2024-04-26T22:27:20.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004a20 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed4161d00 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.411+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41622c0 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.412+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed406a360 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.413+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ec801b070 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.413+0000 7f8ed17fa640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.414+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec808ae20 con 0x7f8ed40d22e0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.414+0000 7f8ed3fff640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.414+0000 7f8ed3fff640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 0x7f8eb806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4148550 tx=0x7f8ec0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.418+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ec80592e0 con 0x7f8ed40d22e0 2024-04-26T22:27:20.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.600+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ed40d0ee0 con 0x7f8eb806c8f0 2024-04-26T22:27:20.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.606+0000 7f8ed17fa640 1 -- 172.21.15.62:0/1291116452 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f8ed40d0ee0 con 0x7f8eb806c8f0 2024-04-26T22:27:20.609 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:20.610 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 msgr2=0x7f8eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 0x7f8eb806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4148550 tx=0x7f8ec0008040 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 msgr2=0x7f8ed415b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800c500 tx=0x7f8ec800c9c0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 shutdown_connections 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8eb806c8f0 0x7f8eb806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ed415f760 0x7f8ed415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed415f380 0x7f8ed406d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 --2- 172.21.15.62:0/1291116452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ed40d22e0 0x7f8ed415b6d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 >> 172.21.15.62:0/1291116452 conn(0x7f8ed409e560 msgr2=0x7f8ed415d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 shutdown_connections 2024-04-26T22:27:20.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.610+0000 7f8edaa7e640 1 -- 172.21.15.62:0/1291116452 wait complete. 2024-04-26T22:27:20.621 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-26T22:27:19.639079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.792300"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985465+0000","last_change":"2024-04-26T22:27:13.884599+0000","last_active":"2024-04-26T22:27:14.985465+0000","last_peered":"2024-04-26T22:27:14.985465+0000","last_clean":"2024-04-26T22:27:14.985465+0000","last_became_active":"2024-04-26T22:27:13.884227+0000","last_became_peered":"2024-04-26T22:27:13.884227+0000","last_unstale":"2024-04-26T22:27:14.985465+0000","last_undegraded":"2024-04-26T22:27:14.985465+0000","last_fullsized":"2024-04-26T22:27:14.985465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:12.033642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861849+0000","last_change":"2024-04-26T22:27:13.869959+0000","last_active":"2024-04-26T22:27:14.861849+0000","last_peered":"2024-04-26T22:27:14.861849+0000","last_clean":"2024-04-26T22:27:14.861849+0000","last_became_active":"2024-04-26T22:27:13.869703+0000","last_became_peered":"2024-04-26T22:27:13.869703+0000","last_unstale":"2024-04-26T22:27:14.861849+0000","last_undegraded":"2024-04-26T22:27:14.861849+0000","last_fullsized":"2024-04-26T22:27:14.861849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985260+0000","last_change":"2024-04-26T22:27:13.868365+0000","last_active":"2024-04-26T22:27:14.985260+0000","last_peered":"2024-04-26T22:27:14.985260+0000","last_clean":"2024-04-26T22:27:14.985260+0000","last_became_active":"2024-04-26T22:27:13.868069+0000","last_became_peered":"2024-04-26T22:27:13.868069+0000","last_unstale":"2024-04-26T22:27:14.985260+0000","last_undegraded":"2024-04-26T22:27:14.985260+0000","last_fullsized":"2024-04-26T22:27:14.985260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864150+0000","last_change":"2024-04-26T22:27:13.883158+0000","last_active":"2024-04-26T22:27:14.864150+0000","last_peered":"2024-04-26T22:27:14.864150+0000","last_clean":"2024-04-26T22:27:14.864150+0000","last_became_active":"2024-04-26T22:27:13.882738+0000","last_became_peered":"2024-04-26T22:27:13.882738+0000","last_unstale":"2024-04-26T22:27:14.864150+0000","last_undegraded":"2024-04-26T22:27:14.864150+0000","last_fullsized":"2024-04-26T22:27:14.864150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985094+0000","last_change":"2024-04-26T22:27:13.881003+0000","last_active":"2024-04-26T22:27:14.985094+0000","last_peered":"2024-04-26T22:27:14.985094+0000","last_clean":"2024-04-26T22:27:14.985094+0000","last_became_active":"2024-04-26T22:27:13.879581+0000","last_became_peered":"2024-04-26T22:27:13.879581+0000","last_unstale":"2024-04-26T22:27:14.985094+0000","last_undegraded":"2024-04-26T22:27:14.985094+0000","last_fullsized":"2024-04-26T22:27:14.985094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864095+0000","last_change":"2024-04-26T22:27:13.886150+0000","last_active":"2024-04-26T22:27:14.864095+0000","last_peered":"2024-04-26T22:27:14.864095+0000","last_clean":"2024-04-26T22:27:14.864095+0000","last_became_active":"2024-04-26T22:27:13.885275+0000","last_became_peered":"2024-04-26T22:27:13.885275+0000","last_unstale":"2024-04-26T22:27:14.864095+0000","last_undegraded":"2024-04-26T22:27:14.864095+0000","last_fullsized":"2024-04-26T22:27:14.864095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984951+0000","last_change":"2024-04-26T22:27:13.879538+0000","last_active":"2024-04-26T22:27:14.984951+0000","last_peered":"2024-04-26T22:27:14.984951+0000","last_clean":"2024-04-26T22:27:14.984951+0000","last_became_active":"2024-04-26T22:27:13.879204+0000","last_became_peered":"2024-04-26T22:27:13.879204+0000","last_unstale":"2024-04-26T22:27:14.984951+0000","last_undegraded":"2024-04-26T22:27:14.984951+0000","last_fullsized":"2024-04-26T22:27:14.984951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985834+0000","last_change":"2024-04-26T22:27:13.887610+0000","last_active":"2024-04-26T22:27:14.985834+0000","last_peered":"2024-04-26T22:27:14.985834+0000","last_clean":"2024-04-26T22:27:14.985834+0000","last_became_active":"2024-04-26T22:27:13.887254+0000","last_became_peered":"2024-04-26T22:27:13.887254+0000","last_unstale":"2024-04-26T22:27:14.985834+0000","last_undegraded":"2024-04-26T22:27:14.985834+0000","last_fullsized":"2024-04-26T22:27:14.985834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985485+0000","last_change":"2024-04-26T22:27:13.889269+0000","last_active":"2024-04-26T22:27:14.985485+0000","last_peered":"2024-04-26T22:27:14.985485+0000","last_clean":"2024-04-26T22:27:14.985485+0000","last_became_active":"2024-04-26T22:27:13.888723+0000","last_became_peered":"2024-04-26T22:27:13.888723+0000","last_unstale":"2024-04-26T22:27:14.985485+0000","last_undegraded":"2024-04-26T22:27:14.985485+0000","last_fullsized":"2024-04-26T22:27:14.985485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:10.483729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879899+0000","last_change":"2024-04-26T22:27:13.888770+0000","last_active":"2024-04-26T22:27:14.879899+0000","last_peered":"2024-04-26T22:27:14.879899+0000","last_clean":"2024-04-26T22:27:14.879899+0000","last_became_active":"2024-04-26T22:27:13.888484+0000","last_became_peered":"2024-04-26T22:27:13.888484+0000","last_unstale":"2024-04-26T22:27:14.879899+0000","last_undegraded":"2024-04-26T22:27:14.879899+0000","last_fullsized":"2024-04-26T22:27:14.879899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879762+0000","last_change":"2024-04-26T22:27:13.889012+0000","last_active":"2024-04-26T22:27:14.879762+0000","last_peered":"2024-04-26T22:27:14.879762+0000","last_clean":"2024-04-26T22:27:14.879762+0000","last_became_active":"2024-04-26T22:27:13.888552+0000","last_became_peered":"2024-04-26T22:27:13.888552+0000","last_unstale":"2024-04-26T22:27:14.879762+0000","last_undegraded":"2024-04-26T22:27:14.879762+0000","last_fullsized":"2024-04-26T22:27:14.879762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863018+0000","last_change":"2024-04-26T22:27:13.883636+0000","last_active":"2024-04-26T22:27:14.863018+0000","last_peered":"2024-04-26T22:27:14.863018+0000","last_clean":"2024-04-26T22:27:14.863018+0000","last_became_active":"2024-04-26T22:27:13.883367+0000","last_became_peered":"2024-04-26T22:27:13.883367+0000","last_unstale":"2024-04-26T22:27:14.863018+0000","last_undegraded":"2024-04-26T22:27:14.863018+0000","last_fullsized":"2024-04-26T22:27:14.863018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:22.436979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985852+0000","last_change":"2024-04-26T22:27:13.883693+0000","last_active":"2024-04-26T22:27:14.985852+0000","last_peered":"2024-04-26T22:27:14.985852+0000","last_clean":"2024-04-26T22:27:14.985852+0000","last_became_active":"2024-04-26T22:27:13.883428+0000","last_became_peered":"2024-04-26T22:27:13.883428+0000","last_unstale":"2024-04-26T22:27:14.985852+0000","last_undegraded":"2024-04-26T22:27:14.985852+0000","last_fullsized":"2024-04-26T22:27:14.985852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984950+0000","last_change":"2024-04-26T22:27:13.867843+0000","last_active":"2024-04-26T22:27:14.984950+0000","last_peered":"2024-04-26T22:27:14.984950+0000","last_clean":"2024-04-26T22:27:14.984950+0000","last_became_active":"2024-04-26T22:27:13.867571+0000","last_became_peered":"2024-04-26T22:27:13.867571+0000","last_unstale":"2024-04-26T22:27:14.984950+0000","last_undegraded":"2024-04-26T22:27:14.984950+0000","last_fullsized":"2024-04-26T22:27:14.984950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.442051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861598+0000","last_change":"2024-04-26T22:27:13.870228+0000","last_active":"2024-04-26T22:27:14.861598+0000","last_peered":"2024-04-26T22:27:14.861598+0000","last_clean":"2024-04-26T22:27:14.861598+0000","last_became_active":"2024-04-26T22:27:13.869325+0000","last_became_peered":"2024-04-26T22:27:13.869325+0000","last_unstale":"2024-04-26T22:27:14.861598+0000","last_undegraded":"2024-04-26T22:27:14.861598+0000","last_fullsized":"2024-04-26T22:27:14.861598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862767+0000","last_change":"2024-04-26T22:27:13.886144+0000","last_active":"2024-04-26T22:27:14.862767+0000","last_peered":"2024-04-26T22:27:14.862767+0000","last_clean":"2024-04-26T22:27:14.862767+0000","last_became_active":"2024-04-26T22:27:13.884929+0000","last_became_peered":"2024-04-26T22:27:13.884929+0000","last_unstale":"2024-04-26T22:27:14.862767+0000","last_undegraded":"2024-04-26T22:27:14.862767+0000","last_fullsized":"2024-04-26T22:27:14.862767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863960+0000","last_change":"2024-04-26T22:27:13.880290+0000","last_active":"2024-04-26T22:27:14.863960+0000","last_peered":"2024-04-26T22:27:14.863960+0000","last_clean":"2024-04-26T22:27:14.863960+0000","last_became_active":"2024-04-26T22:27:13.879946+0000","last_became_peered":"2024-04-26T22:27:13.879946+0000","last_unstale":"2024-04-26T22:27:14.863960+0000","last_undegraded":"2024-04-26T22:27:14.863960+0000","last_fullsized":"2024-04-26T22:27:14.863960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:13.885781+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:13.885512+0000","last_became_peered":"2024-04-26T22:27:13.885512+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985377+0000","last_change":"2024-04-26T22:27:13.880900+0000","last_active":"2024-04-26T22:27:14.985377+0000","last_peered":"2024-04-26T22:27:14.985377+0000","last_clean":"2024-04-26T22:27:14.985377+0000","last_became_active":"2024-04-26T22:27:13.879979+0000","last_became_peered":"2024-04-26T22:27:13.879979+0000","last_unstale":"2024-04-26T22:27:14.985377+0000","last_undegraded":"2024-04-26T22:27:14.985377+0000","last_fullsized":"2024-04-26T22:27:14.985377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879817+0000","last_change":"2024-04-26T22:27:13.882410+0000","last_active":"2024-04-26T22:27:14.879817+0000","last_peered":"2024-04-26T22:27:14.879817+0000","last_clean":"2024-04-26T22:27:14.879817+0000","last_became_active":"2024-04-26T22:27:13.882094+0000","last_became_peered":"2024-04-26T22:27:13.882094+0000","last_unstale":"2024-04-26T22:27:14.879817+0000","last_undegraded":"2024-04-26T22:27:14.879817+0000","last_fullsized":"2024-04-26T22:27:14.879817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985068+0000","last_change":"2024-04-26T22:27:13.880347+0000","last_active":"2024-04-26T22:27:14.985068+0000","last_peered":"2024-04-26T22:27:14.985068+0000","last_clean":"2024-04-26T22:27:14.985068+0000","last_became_active":"2024-04-26T22:27:13.879886+0000","last_became_peered":"2024-04-26T22:27:13.879886+0000","last_unstale":"2024-04-26T22:27:14.985068+0000","last_undegraded":"2024-04-26T22:27:14.985068+0000","last_fullsized":"2024-04-26T22:27:14.985068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863180+0000","last_change":"2024-04-26T22:27:13.887537+0000","last_active":"2024-04-26T22:27:14.863180+0000","last_peered":"2024-04-26T22:27:14.863180+0000","last_clean":"2024-04-26T22:27:14.863180+0000","last_became_active":"2024-04-26T22:27:13.884918+0000","last_became_peered":"2024-04-26T22:27:13.884918+0000","last_unstale":"2024-04-26T22:27:14.863180+0000","last_undegraded":"2024-04-26T22:27:14.863180+0000","last_fullsized":"2024-04-26T22:27:14.863180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985665+0000","last_change":"2024-04-26T22:27:13.871027+0000","last_active":"2024-04-26T22:27:14.985665+0000","last_peered":"2024-04-26T22:27:14.985665+0000","last_clean":"2024-04-26T22:27:14.985665+0000","last_became_active":"2024-04-26T22:27:13.870744+0000","last_became_peered":"2024-04-26T22:27:13.870744+0000","last_unstale":"2024-04-26T22:27:14.985665+0000","last_undegraded":"2024-04-26T22:27:14.985665+0000","last_fullsized":"2024-04-26T22:27:14.985665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862976+0000","last_change":"2024-04-26T22:27:13.881627+0000","last_active":"2024-04-26T22:27:14.862976+0000","last_peered":"2024-04-26T22:27:14.862976+0000","last_clean":"2024-04-26T22:27:14.862976+0000","last_became_active":"2024-04-26T22:27:13.881347+0000","last_became_peered":"2024-04-26T22:27:13.881347+0000","last_unstale":"2024-04-26T22:27:14.862976+0000","last_undegraded":"2024-04-26T22:27:14.862976+0000","last_fullsized":"2024-04-26T22:27:14.862976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:13.869549+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:13.869285+0000","last_became_peered":"2024-04-26T22:27:13.869285+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879982+0000","last_change":"2024-04-26T22:27:13.882698+0000","last_active":"2024-04-26T22:27:14.879982+0000","last_peered":"2024-04-26T22:27:14.879982+0000","last_clean":"2024-04-26T22:27:14.879982+0000","last_became_active":"2024-04-26T22:27:13.882439+0000","last_became_peered":"2024-04-26T22:27:13.882439+0000","last_unstale":"2024-04-26T22:27:14.879982+0000","last_undegraded":"2024-04-26T22:27:14.879982+0000","last_fullsized":"2024-04-26T22:27:14.879982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861681+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.861681+0000","last_peered":"2024-04-26T22:27:14.861681+0000","last_clean":"2024-04-26T22:27:14.861681+0000","last_became_active":"2024-04-26T22:27:13.870898+0000","last_became_peered":"2024-04-26T22:27:13.870898+0000","last_unstale":"2024-04-26T22:27:14.861681+0000","last_undegraded":"2024-04-26T22:27:14.861681+0000","last_fullsized":"2024-04-26T22:27:14.861681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986038+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.986038+0000","last_peered":"2024-04-26T22:27:14.986038+0000","last_clean":"2024-04-26T22:27:14.986038+0000","last_became_active":"2024-04-26T22:27:13.870782+0000","last_became_peered":"2024-04-26T22:27:13.870782+0000","last_unstale":"2024-04-26T22:27:14.986038+0000","last_undegraded":"2024-04-26T22:27:14.986038+0000","last_fullsized":"2024-04-26T22:27:14.986038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985309+0000","last_change":"2024-04-26T22:27:13.873858+0000","last_active":"2024-04-26T22:27:14.985309+0000","last_peered":"2024-04-26T22:27:14.985309+0000","last_clean":"2024-04-26T22:27:14.985309+0000","last_became_active":"2024-04-26T22:27:13.873561+0000","last_became_peered":"2024-04-26T22:27:13.873561+0000","last_unstale":"2024-04-26T22:27:14.985309+0000","last_undegraded":"2024-04-26T22:27:14.985309+0000","last_fullsized":"2024-04-26T22:27:14.985309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863353+0000","last_change":"2024-04-26T22:27:13.885617+0000","last_active":"2024-04-26T22:27:14.863353+0000","last_peered":"2024-04-26T22:27:14.863353+0000","last_clean":"2024-04-26T22:27:14.863353+0000","last_became_active":"2024-04-26T22:27:13.884803+0000","last_became_peered":"2024-04-26T22:27:13.884803+0000","last_unstale":"2024-04-26T22:27:14.863353+0000","last_undegraded":"2024-04-26T22:27:14.863353+0000","last_fullsized":"2024-04-26T22:27:14.863353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984684+0000","last_change":"2024-04-26T22:27:13.885893+0000","last_active":"2024-04-26T22:27:14.984684+0000","last_peered":"2024-04-26T22:27:14.984684+0000","last_clean":"2024-04-26T22:27:14.984684+0000","last_became_active":"2024-04-26T22:27:13.885630+0000","last_became_peered":"2024-04-26T22:27:13.885630+0000","last_unstale":"2024-04-26T22:27:14.984684+0000","last_undegraded":"2024-04-26T22:27:14.984684+0000","last_fullsized":"2024-04-26T22:27:14.984684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879630+0000","last_change":"2024-04-26T22:27:13.887046+0000","last_active":"2024-04-26T22:27:14.879630+0000","last_peered":"2024-04-26T22:27:14.879630+0000","last_clean":"2024-04-26T22:27:14.879630+0000","last_became_active":"2024-04-26T22:27:13.886776+0000","last_became_peered":"2024-04-26T22:27:13.886776+0000","last_unstale":"2024-04-26T22:27:14.879630+0000","last_undegraded":"2024-04-26T22:27:14.879630+0000","last_fullsized":"2024-04-26T22:27:14.879630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863864+0000","last_change":"2024-04-26T22:27:13.877833+0000","last_active":"2024-04-26T22:27:14.863864+0000","last_peered":"2024-04-26T22:27:14.863864+0000","last_clean":"2024-04-26T22:27:14.863864+0000","last_became_active":"2024-04-26T22:27:13.877564+0000","last_became_peered":"2024-04-26T22:27:13.877564+0000","last_unstale":"2024-04-26T22:27:14.863864+0000","last_undegraded":"2024-04-26T22:27:14.863864+0000","last_fullsized":"2024-04-26T22:27:14.863864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985909+0000","last_change":"2024-04-26T22:27:12.870169+0000","last_active":"2024-04-26T22:27:14.985909+0000","last_peered":"2024-04-26T22:27:14.985909+0000","last_clean":"2024-04-26T22:27:14.985909+0000","last_became_active":"2024-04-26T22:27:12.866498+0000","last_became_peered":"2024-04-26T22:27:12.866498+0000","last_unstale":"2024-04-26T22:27:14.985909+0000","last_undegraded":"2024-04-26T22:27:14.985909+0000","last_fullsized":"2024-04-26T22:27:14.985909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985270+0000","last_change":"2024-04-26T22:27:13.870477+0000","last_active":"2024-04-26T22:27:14.985270+0000","last_peered":"2024-04-26T22:27:14.985270+0000","last_clean":"2024-04-26T22:27:14.985270+0000","last_became_active":"2024-04-26T22:27:13.870203+0000","last_became_peered":"2024-04-26T22:27:13.870203+0000","last_unstale":"2024-04-26T22:27:14.985270+0000","last_undegraded":"2024-04-26T22:27:14.985270+0000","last_fullsized":"2024-04-26T22:27:14.985270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879610+0000","last_change":"2024-04-26T22:27:12.866037+0000","last_active":"2024-04-26T22:27:14.879610+0000","last_peered":"2024-04-26T22:27:14.879610+0000","last_clean":"2024-04-26T22:27:14.879610+0000","last_became_active":"2024-04-26T22:27:12.865735+0000","last_became_peered":"2024-04-26T22:27:12.865735+0000","last_unstale":"2024-04-26T22:27:14.879610+0000","last_undegraded":"2024-04-26T22:27:14.879610+0000","last_fullsized":"2024-04-26T22:27:14.879610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985764+0000","last_change":"2024-04-26T22:27:12.870843+0000","last_active":"2024-04-26T22:27:14.985764+0000","last_peered":"2024-04-26T22:27:14.985764+0000","last_clean":"2024-04-26T22:27:14.985764+0000","last_became_active":"2024-04-26T22:27:12.870301+0000","last_became_peered":"2024-04-26T22:27:12.870301+0000","last_unstale":"2024-04-26T22:27:14.985764+0000","last_undegraded":"2024-04-26T22:27:14.985764+0000","last_fullsized":"2024-04-26T22:27:14.985764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.836867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985619+0000","last_change":"2024-04-26T22:27:13.871910+0000","last_active":"2024-04-26T22:27:14.985619+0000","last_peered":"2024-04-26T22:27:14.985619+0000","last_clean":"2024-04-26T22:27:14.985619+0000","last_became_active":"2024-04-26T22:27:13.871645+0000","last_became_peered":"2024-04-26T22:27:13.871645+0000","last_unstale":"2024-04-26T22:27:14.985619+0000","last_undegraded":"2024-04-26T22:27:14.985619+0000","last_fullsized":"2024-04-26T22:27:14.985619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863081+0000","last_change":"2024-04-26T22:27:12.860052+0000","last_active":"2024-04-26T22:27:14.863081+0000","last_peered":"2024-04-26T22:27:14.863081+0000","last_clean":"2024-04-26T22:27:14.863081+0000","last_became_active":"2024-04-26T22:27:12.859367+0000","last_became_peered":"2024-04-26T22:27:12.859367+0000","last_unstale":"2024-04-26T22:27:14.863081+0000","last_undegraded":"2024-04-26T22:27:14.863081+0000","last_fullsized":"2024-04-26T22:27:14.863081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861594+0000","last_change":"2024-04-26T22:27:13.868468+0000","last_active":"2024-04-26T22:27:14.861594+0000","last_peered":"2024-04-26T22:27:14.861594+0000","last_clean":"2024-04-26T22:27:14.861594+0000","last_became_active":"2024-04-26T22:27:13.868172+0000","last_became_peered":"2024-04-26T22:27:13.868172+0000","last_unstale":"2024-04-26T22:27:14.861594+0000","last_undegraded":"2024-04-26T22:27:14.861594+0000","last_fullsized":"2024-04-26T22:27:14.861594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879909+0000","last_change":"2024-04-26T22:27:12.865874+0000","last_active":"2024-04-26T22:27:14.879909+0000","last_peered":"2024-04-26T22:27:14.879909+0000","last_clean":"2024-04-26T22:27:14.879909+0000","last_became_active":"2024-04-26T22:27:12.865593+0000","last_became_peered":"2024-04-26T22:27:12.865593+0000","last_unstale":"2024-04-26T22:27:14.879909+0000","last_undegraded":"2024-04-26T22:27:14.879909+0000","last_fullsized":"2024-04-26T22:27:14.879909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863394+0000","last_change":"2024-04-26T22:27:13.881094+0000","last_active":"2024-04-26T22:27:14.863394+0000","last_peered":"2024-04-26T22:27:14.863394+0000","last_clean":"2024-04-26T22:27:14.863394+0000","last_became_active":"2024-04-26T22:27:13.880021+0000","last_became_peered":"2024-04-26T22:27:13.880021+0000","last_unstale":"2024-04-26T22:27:14.863394+0000","last_undegraded":"2024-04-26T22:27:14.863394+0000","last_fullsized":"2024-04-26T22:27:14.863394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863254+0000","last_change":"2024-04-26T22:27:12.859862+0000","last_active":"2024-04-26T22:27:14.863254+0000","last_peered":"2024-04-26T22:27:14.863254+0000","last_clean":"2024-04-26T22:27:14.863254+0000","last_became_active":"2024-04-26T22:27:12.859130+0000","last_became_peered":"2024-04-26T22:27:12.859130+0000","last_unstale":"2024-04-26T22:27:14.863254+0000","last_undegraded":"2024-04-26T22:27:14.863254+0000","last_fullsized":"2024-04-26T22:27:14.863254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985290+0000","last_change":"2024-04-26T22:27:13.876436+0000","last_active":"2024-04-26T22:27:14.985290+0000","last_peered":"2024-04-26T22:27:14.985290+0000","last_clean":"2024-04-26T22:27:14.985290+0000","last_became_active":"2024-04-26T22:27:13.876123+0000","last_became_peered":"2024-04-26T22:27:13.876123+0000","last_unstale":"2024-04-26T22:27:14.985290+0000","last_undegraded":"2024-04-26T22:27:14.985290+0000","last_fullsized":"2024-04-26T22:27:14.985290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861670+0000","last_change":"2024-04-26T22:27:12.864622+0000","last_active":"2024-04-26T22:27:14.861670+0000","last_peered":"2024-04-26T22:27:14.861670+0000","last_clean":"2024-04-26T22:27:14.861670+0000","last_became_active":"2024-04-26T22:27:12.864324+0000","last_became_peered":"2024-04-26T22:27:12.864324+0000","last_unstale":"2024-04-26T22:27:14.861670+0000","last_undegraded":"2024-04-26T22:27:14.861670+0000","last_fullsized":"2024-04-26T22:27:14.861670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863255+0000","last_change":"2024-04-26T22:27:13.874206+0000","last_active":"2024-04-26T22:27:14.863255+0000","last_peered":"2024-04-26T22:27:14.863255+0000","last_clean":"2024-04-26T22:27:14.863255+0000","last_became_active":"2024-04-26T22:27:13.873944+0000","last_became_peered":"2024-04-26T22:27:13.873944+0000","last_unstale":"2024-04-26T22:27:14.863255+0000","last_undegraded":"2024-04-26T22:27:14.863255+0000","last_fullsized":"2024-04-26T22:27:14.863255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:17.274394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879809+0000","last_change":"2024-04-26T22:27:12.865709+0000","last_active":"2024-04-26T22:27:14.879809+0000","last_peered":"2024-04-26T22:27:14.879809+0000","last_clean":"2024-04-26T22:27:14.879809+0000","last_became_active":"2024-04-26T22:27:12.865398+0000","last_became_peered":"2024-04-26T22:27:12.865398+0000","last_unstale":"2024-04-26T22:27:14.879809+0000","last_undegraded":"2024-04-26T22:27:14.879809+0000","last_fullsized":"2024-04-26T22:27:14.879809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985173+0000","last_change":"2024-04-26T22:27:13.886280+0000","last_active":"2024-04-26T22:27:14.985173+0000","last_peered":"2024-04-26T22:27:14.985173+0000","last_clean":"2024-04-26T22:27:14.985173+0000","last_became_active":"2024-04-26T22:27:13.886020+0000","last_became_peered":"2024-04-26T22:27:13.886020+0000","last_unstale":"2024-04-26T22:27:14.985173+0000","last_undegraded":"2024-04-26T22:27:14.985173+0000","last_fullsized":"2024-04-26T22:27:14.985173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985378+0000","last_change":"2024-04-26T22:27:12.864771+0000","last_active":"2024-04-26T22:27:14.985378+0000","last_peered":"2024-04-26T22:27:14.985378+0000","last_clean":"2024-04-26T22:27:14.985378+0000","last_became_active":"2024-04-26T22:27:12.864215+0000","last_became_peered":"2024-04-26T22:27:12.864215+0000","last_unstale":"2024-04-26T22:27:14.985378+0000","last_undegraded":"2024-04-26T22:27:14.985378+0000","last_fullsized":"2024-04-26T22:27:14.985378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:09.919428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984993+0000","last_change":"2024-04-26T22:27:13.877703+0000","last_active":"2024-04-26T22:27:14.984993+0000","last_peered":"2024-04-26T22:27:14.984993+0000","last_clean":"2024-04-26T22:27:14.984993+0000","last_became_active":"2024-04-26T22:27:13.877406+0000","last_became_peered":"2024-04-26T22:27:13.877406+0000","last_unstale":"2024-04-26T22:27:14.984993+0000","last_undegraded":"2024-04-26T22:27:14.984993+0000","last_fullsized":"2024-04-26T22:27:14.984993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.849288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863454+0000","last_change":"2024-04-26T22:27:12.860680+0000","last_active":"2024-04-26T22:27:14.863454+0000","last_peered":"2024-04-26T22:27:14.863454+0000","last_clean":"2024-04-26T22:27:14.863454+0000","last_became_active":"2024-04-26T22:27:12.860342+0000","last_became_peered":"2024-04-26T22:27:12.860342+0000","last_unstale":"2024-04-26T22:27:14.863454+0000","last_undegraded":"2024-04-26T22:27:14.863454+0000","last_fullsized":"2024-04-26T22:27:14.863454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985217+0000","last_change":"2024-04-26T22:27:13.887857+0000","last_active":"2024-04-26T22:27:14.985217+0000","last_peered":"2024-04-26T22:27:14.985217+0000","last_clean":"2024-04-26T22:27:14.985217+0000","last_became_active":"2024-04-26T22:27:13.887578+0000","last_became_peered":"2024-04-26T22:27:13.887578+0000","last_unstale":"2024-04-26T22:27:14.985217+0000","last_undegraded":"2024-04-26T22:27:14.985217+0000","last_fullsized":"2024-04-26T22:27:14.985217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879621+0000","last_change":"2024-04-26T22:27:12.863476+0000","last_active":"2024-04-26T22:27:14.879621+0000","last_peered":"2024-04-26T22:27:14.879621+0000","last_clean":"2024-04-26T22:27:14.879621+0000","last_became_active":"2024-04-26T22:27:12.863172+0000","last_became_peered":"2024-04-26T22:27:12.863172+0000","last_unstale":"2024-04-26T22:27:14.879621+0000","last_undegraded":"2024-04-26T22:27:14.879621+0000","last_fullsized":"2024-04-26T22:27:14.879621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879712+0000","last_change":"2024-04-26T22:27:13.885569+0000","last_active":"2024-04-26T22:27:14.879712+0000","last_peered":"2024-04-26T22:27:14.879712+0000","last_clean":"2024-04-26T22:27:14.879712+0000","last_became_active":"2024-04-26T22:27:13.885255+0000","last_became_peered":"2024-04-26T22:27:13.885255+0000","last_unstale":"2024-04-26T22:27:14.879712+0000","last_undegraded":"2024-04-26T22:27:14.879712+0000","last_fullsized":"2024-04-26T22:27:14.879712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984474+0000","last_change":"2024-04-26T22:27:12.856942+0000","last_active":"2024-04-26T22:27:14.984474+0000","last_peered":"2024-04-26T22:27:14.984474+0000","last_clean":"2024-04-26T22:27:14.984474+0000","last_became_active":"2024-04-26T22:27:12.856199+0000","last_became_peered":"2024-04-26T22:27:12.856199+0000","last_unstale":"2024-04-26T22:27:14.984474+0000","last_undegraded":"2024-04-26T22:27:14.984474+0000","last_fullsized":"2024-04-26T22:27:14.984474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984932+0000","last_change":"2024-04-26T22:27:13.888422+0000","last_active":"2024-04-26T22:27:14.984932+0000","last_peered":"2024-04-26T22:27:14.984932+0000","last_clean":"2024-04-26T22:27:14.984932+0000","last_became_active":"2024-04-26T22:27:13.888063+0000","last_became_peered":"2024-04-26T22:27:13.888063+0000","last_unstale":"2024-04-26T22:27:14.984932+0000","last_undegraded":"2024-04-26T22:27:14.984932+0000","last_fullsized":"2024-04-26T22:27:14.984932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863321+0000","last_change":"2024-04-26T22:27:12.863291+0000","last_active":"2024-04-26T22:27:14.863321+0000","last_peered":"2024-04-26T22:27:14.863321+0000","last_clean":"2024-04-26T22:27:14.863321+0000","last_became_active":"2024-04-26T22:27:12.863016+0000","last_became_peered":"2024-04-26T22:27:12.863016+0000","last_unstale":"2024-04-26T22:27:14.863321+0000","last_undegraded":"2024-04-26T22:27:14.863321+0000","last_fullsized":"2024-04-26T22:27:14.863321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985407+0000","last_change":"2024-04-26T22:27:13.880545+0000","last_active":"2024-04-26T22:27:14.985407+0000","last_peered":"2024-04-26T22:27:14.985407+0000","last_clean":"2024-04-26T22:27:14.985407+0000","last_became_active":"2024-04-26T22:27:13.880053+0000","last_became_peered":"2024-04-26T22:27:13.880053+0000","last_unstale":"2024-04-26T22:27:14.985407+0000","last_undegraded":"2024-04-26T22:27:14.985407+0000","last_fullsized":"2024-04-26T22:27:14.985407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984817+0000","last_change":"2024-04-26T22:27:12.856649+0000","last_active":"2024-04-26T22:27:14.984817+0000","last_peered":"2024-04-26T22:27:14.984817+0000","last_clean":"2024-04-26T22:27:14.984817+0000","last_became_active":"2024-04-26T22:27:12.856005+0000","last_became_peered":"2024-04-26T22:27:12.856005+0000","last_unstale":"2024-04-26T22:27:14.984817+0000","last_undegraded":"2024-04-26T22:27:14.984817+0000","last_fullsized":"2024-04-26T22:27:14.984817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985551+0000","last_change":"2024-04-26T22:27:13.888874+0000","last_active":"2024-04-26T22:27:14.985551+0000","last_peered":"2024-04-26T22:27:14.985551+0000","last_clean":"2024-04-26T22:27:14.985551+0000","last_became_active":"2024-04-26T22:27:13.888500+0000","last_became_peered":"2024-04-26T22:27:13.888500+0000","last_unstale":"2024-04-26T22:27:14.985551+0000","last_undegraded":"2024-04-26T22:27:14.985551+0000","last_fullsized":"2024-04-26T22:27:14.985551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:55.006561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861789+0000","last_change":"2024-04-26T22:27:12.863335+0000","last_active":"2024-04-26T22:27:14.861789+0000","last_peered":"2024-04-26T22:27:14.861789+0000","last_clean":"2024-04-26T22:27:14.861789+0000","last_became_active":"2024-04-26T22:27:12.862860+0000","last_became_peered":"2024-04-26T22:27:12.862860+0000","last_unstale":"2024-04-26T22:27:14.861789+0000","last_undegraded":"2024-04-26T22:27:14.861789+0000","last_fullsized":"2024-04-26T22:27:14.861789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864321+0000","last_change":"2024-04-26T22:27:13.882256+0000","last_active":"2024-04-26T22:27:14.864321+0000","last_peered":"2024-04-26T22:27:14.864321+0000","last_clean":"2024-04-26T22:27:14.864321+0000","last_became_active":"2024-04-26T22:27:13.882007+0000","last_became_peered":"2024-04-26T22:27:13.882007+0000","last_unstale":"2024-04-26T22:27:14.864321+0000","last_undegraded":"2024-04-26T22:27:14.864321+0000","last_fullsized":"2024-04-26T22:27:14.864321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984706+0000","last_change":"2024-04-26T22:27:12.866195+0000","last_active":"2024-04-26T22:27:14.984706+0000","last_peered":"2024-04-26T22:27:14.984706+0000","last_clean":"2024-04-26T22:27:14.984706+0000","last_became_active":"2024-04-26T22:27:12.865877+0000","last_became_peered":"2024-04-26T22:27:12.865877+0000","last_unstale":"2024-04-26T22:27:14.984706+0000","last_undegraded":"2024-04-26T22:27:14.984706+0000","last_fullsized":"2024-04-26T22:27:14.984706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985189+0000","last_change":"2024-04-26T22:27:13.887105+0000","last_active":"2024-04-26T22:27:14.985189+0000","last_peered":"2024-04-26T22:27:14.985189+0000","last_clean":"2024-04-26T22:27:14.985189+0000","last_became_active":"2024-04-26T22:27:13.886847+0000","last_became_peered":"2024-04-26T22:27:13.886847+0000","last_unstale":"2024-04-26T22:27:14.985189+0000","last_undegraded":"2024-04-26T22:27:14.985189+0000","last_fullsized":"2024-04-26T22:27:14.985189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984901+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:27:14.984901+0000","last_peered":"2024-04-26T22:27:14.984901+0000","last_clean":"2024-04-26T22:27:14.984901+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:27:14.984901+0000","last_undegraded":"2024-04-26T22:27:14.984901+0000","last_fullsized":"2024-04-26T22:27:14.984901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863214+0000","last_change":"2024-04-26T22:27:12.855353+0000","last_active":"2024-04-26T22:27:14.863214+0000","last_peered":"2024-04-26T22:27:14.863214+0000","last_clean":"2024-04-26T22:27:14.863214+0000","last_became_active":"2024-04-26T22:27:12.854814+0000","last_became_peered":"2024-04-26T22:27:12.854814+0000","last_unstale":"2024-04-26T22:27:14.863214+0000","last_undegraded":"2024-04-26T22:27:14.863214+0000","last_fullsized":"2024-04-26T22:27:14.863214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985768+0000","last_change":"2024-04-26T22:27:13.888534+0000","last_active":"2024-04-26T22:27:14.985768+0000","last_peered":"2024-04-26T22:27:14.985768+0000","last_clean":"2024-04-26T22:27:14.985768+0000","last_became_active":"2024-04-26T22:27:13.888091+0000","last_became_peered":"2024-04-26T22:27:13.888091+0000","last_unstale":"2024-04-26T22:27:14.985768+0000","last_undegraded":"2024-04-26T22:27:14.985768+0000","last_fullsized":"2024-04-26T22:27:14.985768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984835+0000","last_change":"2024-04-26T22:27:12.869665+0000","last_active":"2024-04-26T22:27:14.984835+0000","last_peered":"2024-04-26T22:27:14.984835+0000","last_clean":"2024-04-26T22:27:14.984835+0000","last_became_active":"2024-04-26T22:27:12.869352+0000","last_became_peered":"2024-04-26T22:27:12.869352+0000","last_unstale":"2024-04-26T22:27:14.984835+0000","last_undegraded":"2024-04-26T22:27:14.984835+0000","last_fullsized":"2024-04-26T22:27:14.984835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985026+0000","last_change":"2024-04-26T22:27:13.869644+0000","last_active":"2024-04-26T22:27:14.985026+0000","last_peered":"2024-04-26T22:27:14.985026+0000","last_clean":"2024-04-26T22:27:14.985026+0000","last_became_active":"2024-04-26T22:27:13.869375+0000","last_became_peered":"2024-04-26T22:27:13.869375+0000","last_unstale":"2024-04-26T22:27:14.985026+0000","last_undegraded":"2024-04-26T22:27:14.985026+0000","last_fullsized":"2024-04-26T22:27:14.985026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:39.482073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984581+0000","last_change":"2024-04-26T22:27:12.859680+0000","last_active":"2024-04-26T22:27:14.984581+0000","last_peered":"2024-04-26T22:27:14.984581+0000","last_clean":"2024-04-26T22:27:14.984581+0000","last_became_active":"2024-04-26T22:27:12.857700+0000","last_became_peered":"2024-04-26T22:27:12.857700+0000","last_unstale":"2024-04-26T22:27:14.984581+0000","last_undegraded":"2024-04-26T22:27:14.984581+0000","last_fullsized":"2024-04-26T22:27:14.984581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861737+0000","last_change":"2024-04-26T22:27:13.870073+0000","last_active":"2024-04-26T22:27:14.861737+0000","last_peered":"2024-04-26T22:27:14.861737+0000","last_clean":"2024-04-26T22:27:14.861737+0000","last_became_active":"2024-04-26T22:27:13.869803+0000","last_became_peered":"2024-04-26T22:27:13.869803+0000","last_unstale":"2024-04-26T22:27:14.861737+0000","last_undegraded":"2024-04-26T22:27:14.861737+0000","last_fullsized":"2024-04-26T22:27:14.861737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863550+0000","last_change":"2024-04-26T22:27:12.859881+0000","last_active":"2024-04-26T22:27:14.863550+0000","last_peered":"2024-04-26T22:27:14.863550+0000","last_clean":"2024-04-26T22:27:14.863550+0000","last_became_active":"2024-04-26T22:27:12.859599+0000","last_became_peered":"2024-04-26T22:27:12.859599+0000","last_unstale":"2024-04-26T22:27:14.863550+0000","last_undegraded":"2024-04-26T22:27:14.863550+0000","last_fullsized":"2024-04-26T22:27:14.863550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863194+0000","last_change":"2024-04-26T22:27:13.873492+0000","last_active":"2024-04-26T22:27:14.863194+0000","last_peered":"2024-04-26T22:27:14.863194+0000","last_clean":"2024-04-26T22:27:14.863194+0000","last_became_active":"2024-04-26T22:27:13.873235+0000","last_became_peered":"2024-04-26T22:27:13.873235+0000","last_unstale":"2024-04-26T22:27:14.863194+0000","last_undegraded":"2024-04-26T22:27:14.863194+0000","last_fullsized":"2024-04-26T22:27:14.863194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:57.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861708+0000","last_change":"2024-04-26T22:27:12.863689+0000","last_active":"2024-04-26T22:27:14.861708+0000","last_peered":"2024-04-26T22:27:14.861708+0000","last_clean":"2024-04-26T22:27:14.861708+0000","last_became_active":"2024-04-26T22:27:12.862689+0000","last_became_peered":"2024-04-26T22:27:12.862689+0000","last_unstale":"2024-04-26T22:27:14.861708+0000","last_undegraded":"2024-04-26T22:27:14.861708+0000","last_fullsized":"2024-04-26T22:27:14.861708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863992+0000","last_change":"2024-04-26T22:27:13.882989+0000","last_active":"2024-04-26T22:27:14.863992+0000","last_peered":"2024-04-26T22:27:14.863992+0000","last_clean":"2024-04-26T22:27:14.863992+0000","last_became_active":"2024-04-26T22:27:13.882731+0000","last_became_peered":"2024-04-26T22:27:13.882731+0000","last_unstale":"2024-04-26T22:27:14.863992+0000","last_undegraded":"2024-04-26T22:27:14.863992+0000","last_fullsized":"2024-04-26T22:27:14.863992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985421+0000","last_change":"2024-04-26T22:27:12.869598+0000","last_active":"2024-04-26T22:27:14.985421+0000","last_peered":"2024-04-26T22:27:14.985421+0000","last_clean":"2024-04-26T22:27:14.985421+0000","last_became_active":"2024-04-26T22:27:12.869301+0000","last_became_peered":"2024-04-26T22:27:12.869301+0000","last_unstale":"2024-04-26T22:27:14.985421+0000","last_undegraded":"2024-04-26T22:27:14.985421+0000","last_fullsized":"2024-04-26T22:27:14.985421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861610+0000","last_change":"2024-04-26T22:27:13.868115+0000","last_active":"2024-04-26T22:27:14.861610+0000","last_peered":"2024-04-26T22:27:14.861610+0000","last_clean":"2024-04-26T22:27:14.861610+0000","last_became_active":"2024-04-26T22:27:13.867854+0000","last_became_peered":"2024-04-26T22:27:13.867854+0000","last_unstale":"2024-04-26T22:27:14.861610+0000","last_undegraded":"2024-04-26T22:27:14.861610+0000","last_fullsized":"2024-04-26T22:27:14.861610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879222+0000","last_change":"2024-04-26T22:27:12.867931+0000","last_active":"2024-04-26T22:27:14.879222+0000","last_peered":"2024-04-26T22:27:14.879222+0000","last_clean":"2024-04-26T22:27:14.879222+0000","last_became_active":"2024-04-26T22:27:12.867588+0000","last_became_peered":"2024-04-26T22:27:12.867588+0000","last_unstale":"2024-04-26T22:27:14.879222+0000","last_undegraded":"2024-04-26T22:27:14.879222+0000","last_fullsized":"2024-04-26T22:27:14.879222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986195+0000","last_change":"2024-04-26T22:27:13.877905+0000","last_active":"2024-04-26T22:27:14.986195+0000","last_peered":"2024-04-26T22:27:14.986195+0000","last_clean":"2024-04-26T22:27:14.986195+0000","last_became_active":"2024-04-26T22:27:13.877626+0000","last_became_peered":"2024-04-26T22:27:13.877626+0000","last_unstale":"2024-04-26T22:27:14.986195+0000","last_undegraded":"2024-04-26T22:27:14.986195+0000","last_fullsized":"2024-04-26T22:27:14.986195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879942+0000","last_change":"2024-04-26T22:27:12.866913+0000","last_active":"2024-04-26T22:27:14.879942+0000","last_peered":"2024-04-26T22:27:14.879942+0000","last_clean":"2024-04-26T22:27:14.879942+0000","last_became_active":"2024-04-26T22:27:12.866428+0000","last_became_peered":"2024-04-26T22:27:12.866428+0000","last_unstale":"2024-04-26T22:27:14.879942+0000","last_undegraded":"2024-04-26T22:27:14.879942+0000","last_fullsized":"2024-04-26T22:27:14.879942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985432+0000","last_change":"2024-04-26T22:27:13.867975+0000","last_active":"2024-04-26T22:27:14.985432+0000","last_peered":"2024-04-26T22:27:14.985432+0000","last_clean":"2024-04-26T22:27:14.985432+0000","last_became_active":"2024-04-26T22:27:13.867657+0000","last_became_peered":"2024-04-26T22:27:13.867657+0000","last_unstale":"2024-04-26T22:27:14.985432+0000","last_undegraded":"2024-04-26T22:27:14.985432+0000","last_fullsized":"2024-04-26T22:27:14.985432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:12.870598+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:12.870147+0000","last_became_peered":"2024-04-26T22:27:12.870147+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863751+0000","last_change":"2024-04-26T22:27:13.869914+0000","last_active":"2024-04-26T22:27:14.863751+0000","last_peered":"2024-04-26T22:27:14.863751+0000","last_clean":"2024-04-26T22:27:14.863751+0000","last_became_active":"2024-04-26T22:27:13.869626+0000","last_became_peered":"2024-04-26T22:27:13.869626+0000","last_unstale":"2024-04-26T22:27:14.863751+0000","last_undegraded":"2024-04-26T22:27:14.863751+0000","last_fullsized":"2024-04-26T22:27:14.863751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861709+0000","last_change":"2024-04-26T22:27:12.865574+0000","last_active":"2024-04-26T22:27:14.861709+0000","last_peered":"2024-04-26T22:27:14.861709+0000","last_clean":"2024-04-26T22:27:14.861709+0000","last_became_active":"2024-04-26T22:27:12.865293+0000","last_became_peered":"2024-04-26T22:27:12.865293+0000","last_unstale":"2024-04-26T22:27:14.861709+0000","last_undegraded":"2024-04-26T22:27:14.861709+0000","last_fullsized":"2024-04-26T22:27:14.861709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985793+0000","last_change":"2024-04-26T22:27:13.873495+0000","last_active":"2024-04-26T22:27:14.985793+0000","last_peered":"2024-04-26T22:27:14.985793+0000","last_clean":"2024-04-26T22:27:14.985793+0000","last_became_active":"2024-04-26T22:27:13.873110+0000","last_became_peered":"2024-04-26T22:27:13.873110+0000","last_unstale":"2024-04-26T22:27:14.985793+0000","last_undegraded":"2024-04-26T22:27:14.985793+0000","last_fullsized":"2024-04-26T22:27:14.985793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861881+0000","last_change":"2024-04-26T22:27:12.860253+0000","last_active":"2024-04-26T22:27:14.861881+0000","last_peered":"2024-04-26T22:27:14.861881+0000","last_clean":"2024-04-26T22:27:14.861881+0000","last_became_active":"2024-04-26T22:27:12.859755+0000","last_became_peered":"2024-04-26T22:27:12.859755+0000","last_unstale":"2024-04-26T22:27:14.861881+0000","last_undegraded":"2024-04-26T22:27:14.861881+0000","last_fullsized":"2024-04-26T22:27:14.861881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986034+0000","last_change":"2024-04-26T22:27:13.875140+0000","last_active":"2024-04-26T22:27:14.986034+0000","last_peered":"2024-04-26T22:27:14.986034+0000","last_clean":"2024-04-26T22:27:14.986034+0000","last_became_active":"2024-04-26T22:27:13.872838+0000","last_became_peered":"2024-04-26T22:27:13.872838+0000","last_unstale":"2024-04-26T22:27:14.986034+0000","last_undegraded":"2024-04-26T22:27:14.986034+0000","last_fullsized":"2024-04-26T22:27:14.986034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985243+0000","last_change":"2024-04-26T22:27:12.867920+0000","last_active":"2024-04-26T22:27:14.985243+0000","last_peered":"2024-04-26T22:27:14.985243+0000","last_clean":"2024-04-26T22:27:14.985243+0000","last_became_active":"2024-04-26T22:27:12.867652+0000","last_became_peered":"2024-04-26T22:27:12.867652+0000","last_unstale":"2024-04-26T22:27:14.985243+0000","last_undegraded":"2024-04-26T22:27:14.985243+0000","last_fullsized":"2024-04-26T22:27:14.985243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:36.571464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879782+0000","last_change":"2024-04-26T22:27:13.885168+0000","last_active":"2024-04-26T22:27:14.879782+0000","last_peered":"2024-04-26T22:27:14.879782+0000","last_clean":"2024-04-26T22:27:14.879782+0000","last_became_active":"2024-04-26T22:27:13.884882+0000","last_became_peered":"2024-04-26T22:27:13.884882+0000","last_unstale":"2024-04-26T22:27:14.879782+0000","last_undegraded":"2024-04-26T22:27:14.879782+0000","last_fullsized":"2024-04-26T22:27:14.879782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863833+0000","last_change":"2024-04-26T22:27:13.878648+0000","last_active":"2024-04-26T22:27:14.863833+0000","last_peered":"2024-04-26T22:27:14.863833+0000","last_clean":"2024-04-26T22:27:14.863833+0000","last_became_active":"2024-04-26T22:27:13.878280+0000","last_became_peered":"2024-04-26T22:27:13.878280+0000","last_unstale":"2024-04-26T22:27:14.863833+0000","last_undegraded":"2024-04-26T22:27:14.863833+0000","last_fullsized":"2024-04-26T22:27:14.863833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985097+0000","last_change":"2024-04-26T22:27:12.865334+0000","last_active":"2024-04-26T22:27:14.985097+0000","last_peered":"2024-04-26T22:27:14.985097+0000","last_clean":"2024-04-26T22:27:14.985097+0000","last_became_active":"2024-04-26T22:27:12.865055+0000","last_became_peered":"2024-04-26T22:27:12.865055+0000","last_unstale":"2024-04-26T22:27:14.985097+0000","last_undegraded":"2024-04-26T22:27:14.985097+0000","last_fullsized":"2024-04-26T22:27:14.985097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864217+0000","last_change":"2024-04-26T22:27:13.879604+0000","last_active":"2024-04-26T22:27:14.864217+0000","last_peered":"2024-04-26T22:27:14.864217+0000","last_clean":"2024-04-26T22:27:14.864217+0000","last_became_active":"2024-04-26T22:27:13.879317+0000","last_became_peered":"2024-04-26T22:27:13.879317+0000","last_unstale":"2024-04-26T22:27:14.864217+0000","last_undegraded":"2024-04-26T22:27:14.864217+0000","last_fullsized":"2024-04-26T22:27:14.864217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985330+0000","last_change":"2024-04-26T22:27:12.870741+0000","last_active":"2024-04-26T22:27:14.985330+0000","last_peered":"2024-04-26T22:27:14.985330+0000","last_clean":"2024-04-26T22:27:14.985330+0000","last_became_active":"2024-04-26T22:27:12.870356+0000","last_became_peered":"2024-04-26T22:27:12.870356+0000","last_unstale":"2024-04-26T22:27:14.985330+0000","last_undegraded":"2024-04-26T22:27:14.985330+0000","last_fullsized":"2024-04-26T22:27:14.985330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.747576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:13.888713+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:13.888393+0000","last_became_peered":"2024-04-26T22:27:13.888393+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:12.870271+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:12.869919+0000","last_became_peered":"2024-04-26T22:27:12.869919+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985725+0000","last_change":"2024-04-26T22:27:13.883151+0000","last_active":"2024-04-26T22:27:14.985725+0000","last_peered":"2024-04-26T22:27:14.985725+0000","last_clean":"2024-04-26T22:27:14.985725+0000","last_became_active":"2024-04-26T22:27:13.882875+0000","last_became_peered":"2024-04-26T22:27:13.882875+0000","last_unstale":"2024-04-26T22:27:14.985725+0000","last_undegraded":"2024-04-26T22:27:14.985725+0000","last_fullsized":"2024-04-26T22:27:14.985725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:12.865998+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:12.865683+0000","last_became_peered":"2024-04-26T22:27:12.865683+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:27:20.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:27:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.762+0000 7f0af3946640 1 Processor -- start 2024-04-26T22:27:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.762+0000 7f0af3946640 1 -- start start 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af3946640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0aec0d3820 0x7f0aec0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af3946640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0aec0b4ba0 0x7f0aec0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af3946640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0aec0d0210 con 0x7f0aec162b40 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af3946640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0aec0d41c0 con 0x7f0aec0d3820 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af3946640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0aec0d04f0 con 0x7f0aec0b4ba0 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af0ee1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0aec0d3820 0x7f0aec0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af16e2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0aec162b40 0x7f0aec0d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56466/0 (socket says 172.21.15.62:56466) 2024-04-26T22:27:20.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af16e2640 1 -- 172.21.15.62:0/2439578805 learned_addr learned my addr 172.21.15.62:0/2439578805 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af0ee1640 1 -- 172.21.15.62:0/2439578805 >> v2:172.21.15.174:3300/0 conn(0x7f0aec0b4ba0 msgr2=0x7f0aec0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.763+0000 7f0af0ee1640 1 --2- 172.21.15.62:0/2439578805 >> v2:172.21.15.174:3300/0 conn(0x7f0aec0b4ba0 0x7f0aec0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0af0ee1640 1 -- 172.21.15.62:0/2439578805 >> v1:172.21.15.62:6789/0 conn(0x7f0aec162b40 legacy=0x7f0aec0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0ae3fff640 1 -- 172.21.15.62:0/2439578805 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445334672 0 0) 0x7f0aec0d0210 con 0x7f0aec162b40 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0af0ee1640 1 -- 172.21.15.62:0/2439578805 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec0d07d0 con 0x7f0aec0d3820 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0af0ee1640 1 --2- 172.21.15.62:0/2439578805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec0d3820 0x7f0aec0d3c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0aec09ec60 tx=0x7f0ae802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c346f6e5c34ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0ae3fff640 1 -- 172.21.15.62:0/2439578805 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8037070 con 0x7f0aec0d3820 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0ae3fff640 1 -- 172.21.15.62:0/2439578805 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae8032440 con 0x7f0aec0d3820 2024-04-26T22:27:20.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.764+0000 7f0ae3fff640 1 -- 172.21.15.62:0/2439578805 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8032810 con 0x7f0aec0d3820 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 -- 172.21.15.62:0/2439578805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec0d3820 msgr2=0x7f0aec0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 --2- 172.21.15.62:0/2439578805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec0d3820 0x7f0aec0d3c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0aec09ec60 tx=0x7f0ae802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 -- 172.21.15.62:0/2439578805 shutdown_connections 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 --2- 172.21.15.62:0/2439578805 >> v2:172.21.15.174:3300/0 conn(0x7f0aec0b4ba0 0x7f0aec0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 --2- 172.21.15.62:0/2439578805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec0d3820 0x7f0aec0d3c80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 -- 172.21.15.62:0/2439578805 >> 172.21.15.62:0/2439578805 conn(0x7f0aec09e550 msgr2=0x7f0aec0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 -- 172.21.15.62:0/2439578805 shutdown_connections 2024-04-26T22:27:20.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.765+0000 7f0af3946640 1 -- 172.21.15.62:0/2439578805 wait complete. 2024-04-26T22:27:20.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.766+0000 7f0af3946640 1 Processor -- start 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.766+0000 7f0af3946640 1 -- start start 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aec0b4ba0 0x7f0aec14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 0x7f0aec0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec0d12b0 con 0x7f0aec162b40 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aec0d0d90 con 0x7f0aec0d3820 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af3946640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0aec0cfe30 con 0x7f0aec0b4ba0 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35652/0 (socket says 172.21.15.62:35652) 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 -- 172.21.15.62:0/3565983773 learned_addr learned my addr 172.21.15.62:0/3565983773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af0ee1640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 0x7f0aec0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af16e2640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aec0b4ba0 0x7f0aec14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 -- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aec0b4ba0 msgr2=0x7f0aec14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aec0b4ba0 0x7f0aec14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 -- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 msgr2=0x7f0aec0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 0x7f0aec0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 -- 172.21.15.62:0/3565983773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aec0ce8e0 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af0ee1640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 0x7f0aec0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.767+0000 7f0af1ee3640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b970 tx=0x7f0ae400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.768+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4019070 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.768+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae400cb60 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.768+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aec0cebc0 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.768+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4014440 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.768+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aec0cf180 con 0x7f0aec162b40 2024-04-26T22:27:20.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.769+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aec06a360 con 0x7f0aec162b40 2024-04-26T22:27:20.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.769+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ae401c070 con 0x7f0aec162b40 2024-04-26T22:27:20.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.769+0000 7f0ae1ffb640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 0x7f0ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:20.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.770+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ae408b870 con 0x7f0aec162b40 2024-04-26T22:27:20.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.770+0000 7f0af16e2640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 0x7f0ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:20.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.770+0000 7f0af16e2640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 0x7f0ad006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0adc000f60 tx=0x7f0adc007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:20.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.774+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ae4059d30 con 0x7f0aec162b40 2024-04-26T22:27:20.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.956+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0aec0d2960 con 0x7f0ad006c8f0 2024-04-26T22:27:20.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.962+0000 7f0ae1ffb640 1 -- 172.21.15.62:0/3565983773 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f0aec0d2960 con 0x7f0ad006c8f0 2024-04-26T22:27:20.965 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:20.967 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:27:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 msgr2=0x7f0ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 0x7f0ad006edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0adc000f60 tx=0x7f0adc007b50 comp rx=0 tx=0).stop 2024-04-26T22:27:20.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 msgr2=0x7f0aec1612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b970 tx=0x7f0ae400be30 comp rx=0 tx=0).stop 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 shutdown_connections 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0ad006c8f0 0x7f0ad006edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0aec162b40 0x7f0aec1612b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aec0d3820 0x7f0aec0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 --2- 172.21.15.62:0/3565983773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0aec0b4ba0 0x7f0aec14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 >> 172.21.15.62:0/3565983773 conn(0x7f0aec09e550 msgr2=0x7f0aec06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 shutdown_connections 2024-04-26T22:27:20.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:20.965+0000 7f0af3946640 1 -- 172.21.15.62:0/3565983773 wait complete. 2024-04-26T22:27:20.976 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-26T22:27:19.639079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.792300"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985465+0000","last_change":"2024-04-26T22:27:13.884599+0000","last_active":"2024-04-26T22:27:14.985465+0000","last_peered":"2024-04-26T22:27:14.985465+0000","last_clean":"2024-04-26T22:27:14.985465+0000","last_became_active":"2024-04-26T22:27:13.884227+0000","last_became_peered":"2024-04-26T22:27:13.884227+0000","last_unstale":"2024-04-26T22:27:14.985465+0000","last_undegraded":"2024-04-26T22:27:14.985465+0000","last_fullsized":"2024-04-26T22:27:14.985465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:12.033642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861849+0000","last_change":"2024-04-26T22:27:13.869959+0000","last_active":"2024-04-26T22:27:14.861849+0000","last_peered":"2024-04-26T22:27:14.861849+0000","last_clean":"2024-04-26T22:27:14.861849+0000","last_became_active":"2024-04-26T22:27:13.869703+0000","last_became_peered":"2024-04-26T22:27:13.869703+0000","last_unstale":"2024-04-26T22:27:14.861849+0000","last_undegraded":"2024-04-26T22:27:14.861849+0000","last_fullsized":"2024-04-26T22:27:14.861849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985260+0000","last_change":"2024-04-26T22:27:13.868365+0000","last_active":"2024-04-26T22:27:14.985260+0000","last_peered":"2024-04-26T22:27:14.985260+0000","last_clean":"2024-04-26T22:27:14.985260+0000","last_became_active":"2024-04-26T22:27:13.868069+0000","last_became_peered":"2024-04-26T22:27:13.868069+0000","last_unstale":"2024-04-26T22:27:14.985260+0000","last_undegraded":"2024-04-26T22:27:14.985260+0000","last_fullsized":"2024-04-26T22:27:14.985260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864150+0000","last_change":"2024-04-26T22:27:13.883158+0000","last_active":"2024-04-26T22:27:14.864150+0000","last_peered":"2024-04-26T22:27:14.864150+0000","last_clean":"2024-04-26T22:27:14.864150+0000","last_became_active":"2024-04-26T22:27:13.882738+0000","last_became_peered":"2024-04-26T22:27:13.882738+0000","last_unstale":"2024-04-26T22:27:14.864150+0000","last_undegraded":"2024-04-26T22:27:14.864150+0000","last_fullsized":"2024-04-26T22:27:14.864150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985094+0000","last_change":"2024-04-26T22:27:13.881003+0000","last_active":"2024-04-26T22:27:14.985094+0000","last_peered":"2024-04-26T22:27:14.985094+0000","last_clean":"2024-04-26T22:27:14.985094+0000","last_became_active":"2024-04-26T22:27:13.879581+0000","last_became_peered":"2024-04-26T22:27:13.879581+0000","last_unstale":"2024-04-26T22:27:14.985094+0000","last_undegraded":"2024-04-26T22:27:14.985094+0000","last_fullsized":"2024-04-26T22:27:14.985094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864095+0000","last_change":"2024-04-26T22:27:13.886150+0000","last_active":"2024-04-26T22:27:14.864095+0000","last_peered":"2024-04-26T22:27:14.864095+0000","last_clean":"2024-04-26T22:27:14.864095+0000","last_became_active":"2024-04-26T22:27:13.885275+0000","last_became_peered":"2024-04-26T22:27:13.885275+0000","last_unstale":"2024-04-26T22:27:14.864095+0000","last_undegraded":"2024-04-26T22:27:14.864095+0000","last_fullsized":"2024-04-26T22:27:14.864095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984951+0000","last_change":"2024-04-26T22:27:13.879538+0000","last_active":"2024-04-26T22:27:14.984951+0000","last_peered":"2024-04-26T22:27:14.984951+0000","last_clean":"2024-04-26T22:27:14.984951+0000","last_became_active":"2024-04-26T22:27:13.879204+0000","last_became_peered":"2024-04-26T22:27:13.879204+0000","last_unstale":"2024-04-26T22:27:14.984951+0000","last_undegraded":"2024-04-26T22:27:14.984951+0000","last_fullsized":"2024-04-26T22:27:14.984951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985834+0000","last_change":"2024-04-26T22:27:13.887610+0000","last_active":"2024-04-26T22:27:14.985834+0000","last_peered":"2024-04-26T22:27:14.985834+0000","last_clean":"2024-04-26T22:27:14.985834+0000","last_became_active":"2024-04-26T22:27:13.887254+0000","last_became_peered":"2024-04-26T22:27:13.887254+0000","last_unstale":"2024-04-26T22:27:14.985834+0000","last_undegraded":"2024-04-26T22:27:14.985834+0000","last_fullsized":"2024-04-26T22:27:14.985834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985485+0000","last_change":"2024-04-26T22:27:13.889269+0000","last_active":"2024-04-26T22:27:14.985485+0000","last_peered":"2024-04-26T22:27:14.985485+0000","last_clean":"2024-04-26T22:27:14.985485+0000","last_became_active":"2024-04-26T22:27:13.888723+0000","last_became_peered":"2024-04-26T22:27:13.888723+0000","last_unstale":"2024-04-26T22:27:14.985485+0000","last_undegraded":"2024-04-26T22:27:14.985485+0000","last_fullsized":"2024-04-26T22:27:14.985485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:10.483729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879899+0000","last_change":"2024-04-26T22:27:13.888770+0000","last_active":"2024-04-26T22:27:14.879899+0000","last_peered":"2024-04-26T22:27:14.879899+0000","last_clean":"2024-04-26T22:27:14.879899+0000","last_became_active":"2024-04-26T22:27:13.888484+0000","last_became_peered":"2024-04-26T22:27:13.888484+0000","last_unstale":"2024-04-26T22:27:14.879899+0000","last_undegraded":"2024-04-26T22:27:14.879899+0000","last_fullsized":"2024-04-26T22:27:14.879899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879762+0000","last_change":"2024-04-26T22:27:13.889012+0000","last_active":"2024-04-26T22:27:14.879762+0000","last_peered":"2024-04-26T22:27:14.879762+0000","last_clean":"2024-04-26T22:27:14.879762+0000","last_became_active":"2024-04-26T22:27:13.888552+0000","last_became_peered":"2024-04-26T22:27:13.888552+0000","last_unstale":"2024-04-26T22:27:14.879762+0000","last_undegraded":"2024-04-26T22:27:14.879762+0000","last_fullsized":"2024-04-26T22:27:14.879762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863018+0000","last_change":"2024-04-26T22:27:13.883636+0000","last_active":"2024-04-26T22:27:14.863018+0000","last_peered":"2024-04-26T22:27:14.863018+0000","last_clean":"2024-04-26T22:27:14.863018+0000","last_became_active":"2024-04-26T22:27:13.883367+0000","last_became_peered":"2024-04-26T22:27:13.883367+0000","last_unstale":"2024-04-26T22:27:14.863018+0000","last_undegraded":"2024-04-26T22:27:14.863018+0000","last_fullsized":"2024-04-26T22:27:14.863018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:22.436979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985852+0000","last_change":"2024-04-26T22:27:13.883693+0000","last_active":"2024-04-26T22:27:14.985852+0000","last_peered":"2024-04-26T22:27:14.985852+0000","last_clean":"2024-04-26T22:27:14.985852+0000","last_became_active":"2024-04-26T22:27:13.883428+0000","last_became_peered":"2024-04-26T22:27:13.883428+0000","last_unstale":"2024-04-26T22:27:14.985852+0000","last_undegraded":"2024-04-26T22:27:14.985852+0000","last_fullsized":"2024-04-26T22:27:14.985852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984950+0000","last_change":"2024-04-26T22:27:13.867843+0000","last_active":"2024-04-26T22:27:14.984950+0000","last_peered":"2024-04-26T22:27:14.984950+0000","last_clean":"2024-04-26T22:27:14.984950+0000","last_became_active":"2024-04-26T22:27:13.867571+0000","last_became_peered":"2024-04-26T22:27:13.867571+0000","last_unstale":"2024-04-26T22:27:14.984950+0000","last_undegraded":"2024-04-26T22:27:14.984950+0000","last_fullsized":"2024-04-26T22:27:14.984950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:34.442051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861598+0000","last_change":"2024-04-26T22:27:13.870228+0000","last_active":"2024-04-26T22:27:14.861598+0000","last_peered":"2024-04-26T22:27:14.861598+0000","last_clean":"2024-04-26T22:27:14.861598+0000","last_became_active":"2024-04-26T22:27:13.869325+0000","last_became_peered":"2024-04-26T22:27:13.869325+0000","last_unstale":"2024-04-26T22:27:14.861598+0000","last_undegraded":"2024-04-26T22:27:14.861598+0000","last_fullsized":"2024-04-26T22:27:14.861598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862767+0000","last_change":"2024-04-26T22:27:13.886144+0000","last_active":"2024-04-26T22:27:14.862767+0000","last_peered":"2024-04-26T22:27:14.862767+0000","last_clean":"2024-04-26T22:27:14.862767+0000","last_became_active":"2024-04-26T22:27:13.884929+0000","last_became_peered":"2024-04-26T22:27:13.884929+0000","last_unstale":"2024-04-26T22:27:14.862767+0000","last_undegraded":"2024-04-26T22:27:14.862767+0000","last_fullsized":"2024-04-26T22:27:14.862767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863960+0000","last_change":"2024-04-26T22:27:13.880290+0000","last_active":"2024-04-26T22:27:14.863960+0000","last_peered":"2024-04-26T22:27:14.863960+0000","last_clean":"2024-04-26T22:27:14.863960+0000","last_became_active":"2024-04-26T22:27:13.879946+0000","last_became_peered":"2024-04-26T22:27:13.879946+0000","last_unstale":"2024-04-26T22:27:14.863960+0000","last_undegraded":"2024-04-26T22:27:14.863960+0000","last_fullsized":"2024-04-26T22:27:14.863960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:02.603600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:13.885781+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:13.885512+0000","last_became_peered":"2024-04-26T22:27:13.885512+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985377+0000","last_change":"2024-04-26T22:27:13.880900+0000","last_active":"2024-04-26T22:27:14.985377+0000","last_peered":"2024-04-26T22:27:14.985377+0000","last_clean":"2024-04-26T22:27:14.985377+0000","last_became_active":"2024-04-26T22:27:13.879979+0000","last_became_peered":"2024-04-26T22:27:13.879979+0000","last_unstale":"2024-04-26T22:27:14.985377+0000","last_undegraded":"2024-04-26T22:27:14.985377+0000","last_fullsized":"2024-04-26T22:27:14.985377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879817+0000","last_change":"2024-04-26T22:27:13.882410+0000","last_active":"2024-04-26T22:27:14.879817+0000","last_peered":"2024-04-26T22:27:14.879817+0000","last_clean":"2024-04-26T22:27:14.879817+0000","last_became_active":"2024-04-26T22:27:13.882094+0000","last_became_peered":"2024-04-26T22:27:13.882094+0000","last_unstale":"2024-04-26T22:27:14.879817+0000","last_undegraded":"2024-04-26T22:27:14.879817+0000","last_fullsized":"2024-04-26T22:27:14.879817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985068+0000","last_change":"2024-04-26T22:27:13.880347+0000","last_active":"2024-04-26T22:27:14.985068+0000","last_peered":"2024-04-26T22:27:14.985068+0000","last_clean":"2024-04-26T22:27:14.985068+0000","last_became_active":"2024-04-26T22:27:13.879886+0000","last_became_peered":"2024-04-26T22:27:13.879886+0000","last_unstale":"2024-04-26T22:27:14.985068+0000","last_undegraded":"2024-04-26T22:27:14.985068+0000","last_fullsized":"2024-04-26T22:27:14.985068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863180+0000","last_change":"2024-04-26T22:27:13.887537+0000","last_active":"2024-04-26T22:27:14.863180+0000","last_peered":"2024-04-26T22:27:14.863180+0000","last_clean":"2024-04-26T22:27:14.863180+0000","last_became_active":"2024-04-26T22:27:13.884918+0000","last_became_peered":"2024-04-26T22:27:13.884918+0000","last_unstale":"2024-04-26T22:27:14.863180+0000","last_undegraded":"2024-04-26T22:27:14.863180+0000","last_fullsized":"2024-04-26T22:27:14.863180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985665+0000","last_change":"2024-04-26T22:27:13.871027+0000","last_active":"2024-04-26T22:27:14.985665+0000","last_peered":"2024-04-26T22:27:14.985665+0000","last_clean":"2024-04-26T22:27:14.985665+0000","last_became_active":"2024-04-26T22:27:13.870744+0000","last_became_peered":"2024-04-26T22:27:13.870744+0000","last_unstale":"2024-04-26T22:27:14.985665+0000","last_undegraded":"2024-04-26T22:27:14.985665+0000","last_fullsized":"2024-04-26T22:27:14.985665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.862976+0000","last_change":"2024-04-26T22:27:13.881627+0000","last_active":"2024-04-26T22:27:14.862976+0000","last_peered":"2024-04-26T22:27:14.862976+0000","last_clean":"2024-04-26T22:27:14.862976+0000","last_became_active":"2024-04-26T22:27:13.881347+0000","last_became_peered":"2024-04-26T22:27:13.881347+0000","last_unstale":"2024-04-26T22:27:14.862976+0000","last_undegraded":"2024-04-26T22:27:14.862976+0000","last_fullsized":"2024-04-26T22:27:14.862976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:15.774244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:13.869549+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:13.869285+0000","last_became_peered":"2024-04-26T22:27:13.869285+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879982+0000","last_change":"2024-04-26T22:27:13.882698+0000","last_active":"2024-04-26T22:27:14.879982+0000","last_peered":"2024-04-26T22:27:14.879982+0000","last_clean":"2024-04-26T22:27:14.879982+0000","last_became_active":"2024-04-26T22:27:13.882439+0000","last_became_peered":"2024-04-26T22:27:13.882439+0000","last_unstale":"2024-04-26T22:27:14.879982+0000","last_undegraded":"2024-04-26T22:27:14.879982+0000","last_fullsized":"2024-04-26T22:27:14.879982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861681+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.861681+0000","last_peered":"2024-04-26T22:27:14.861681+0000","last_clean":"2024-04-26T22:27:14.861681+0000","last_became_active":"2024-04-26T22:27:13.870898+0000","last_became_peered":"2024-04-26T22:27:13.870898+0000","last_unstale":"2024-04-26T22:27:14.861681+0000","last_undegraded":"2024-04-26T22:27:14.861681+0000","last_fullsized":"2024-04-26T22:27:14.861681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986038+0000","last_change":"2024-04-26T22:27:13.871199+0000","last_active":"2024-04-26T22:27:14.986038+0000","last_peered":"2024-04-26T22:27:14.986038+0000","last_clean":"2024-04-26T22:27:14.986038+0000","last_became_active":"2024-04-26T22:27:13.870782+0000","last_became_peered":"2024-04-26T22:27:13.870782+0000","last_unstale":"2024-04-26T22:27:14.986038+0000","last_undegraded":"2024-04-26T22:27:14.986038+0000","last_fullsized":"2024-04-26T22:27:14.986038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985309+0000","last_change":"2024-04-26T22:27:13.873858+0000","last_active":"2024-04-26T22:27:14.985309+0000","last_peered":"2024-04-26T22:27:14.985309+0000","last_clean":"2024-04-26T22:27:14.985309+0000","last_became_active":"2024-04-26T22:27:13.873561+0000","last_became_peered":"2024-04-26T22:27:13.873561+0000","last_unstale":"2024-04-26T22:27:14.985309+0000","last_undegraded":"2024-04-26T22:27:14.985309+0000","last_fullsized":"2024-04-26T22:27:14.985309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863353+0000","last_change":"2024-04-26T22:27:13.885617+0000","last_active":"2024-04-26T22:27:14.863353+0000","last_peered":"2024-04-26T22:27:14.863353+0000","last_clean":"2024-04-26T22:27:14.863353+0000","last_became_active":"2024-04-26T22:27:13.884803+0000","last_became_peered":"2024-04-26T22:27:13.884803+0000","last_unstale":"2024-04-26T22:27:14.863353+0000","last_undegraded":"2024-04-26T22:27:14.863353+0000","last_fullsized":"2024-04-26T22:27:14.863353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:09.604424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984684+0000","last_change":"2024-04-26T22:27:13.885893+0000","last_active":"2024-04-26T22:27:14.984684+0000","last_peered":"2024-04-26T22:27:14.984684+0000","last_clean":"2024-04-26T22:27:14.984684+0000","last_became_active":"2024-04-26T22:27:13.885630+0000","last_became_peered":"2024-04-26T22:27:13.885630+0000","last_unstale":"2024-04-26T22:27:14.984684+0000","last_undegraded":"2024-04-26T22:27:14.984684+0000","last_fullsized":"2024-04-26T22:27:14.984684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879630+0000","last_change":"2024-04-26T22:27:13.887046+0000","last_active":"2024-04-26T22:27:14.879630+0000","last_peered":"2024-04-26T22:27:14.879630+0000","last_clean":"2024-04-26T22:27:14.879630+0000","last_became_active":"2024-04-26T22:27:13.886776+0000","last_became_peered":"2024-04-26T22:27:13.886776+0000","last_unstale":"2024-04-26T22:27:14.879630+0000","last_undegraded":"2024-04-26T22:27:14.879630+0000","last_fullsized":"2024-04-26T22:27:14.879630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:41.589679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863864+0000","last_change":"2024-04-26T22:27:13.877833+0000","last_active":"2024-04-26T22:27:14.863864+0000","last_peered":"2024-04-26T22:27:14.863864+0000","last_clean":"2024-04-26T22:27:14.863864+0000","last_became_active":"2024-04-26T22:27:13.877564+0000","last_became_peered":"2024-04-26T22:27:13.877564+0000","last_unstale":"2024-04-26T22:27:14.863864+0000","last_undegraded":"2024-04-26T22:27:14.863864+0000","last_fullsized":"2024-04-26T22:27:14.863864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:36.723650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985909+0000","last_change":"2024-04-26T22:27:12.870169+0000","last_active":"2024-04-26T22:27:14.985909+0000","last_peered":"2024-04-26T22:27:14.985909+0000","last_clean":"2024-04-26T22:27:14.985909+0000","last_became_active":"2024-04-26T22:27:12.866498+0000","last_became_peered":"2024-04-26T22:27:12.866498+0000","last_unstale":"2024-04-26T22:27:14.985909+0000","last_undegraded":"2024-04-26T22:27:14.985909+0000","last_fullsized":"2024-04-26T22:27:14.985909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985270+0000","last_change":"2024-04-26T22:27:13.870477+0000","last_active":"2024-04-26T22:27:14.985270+0000","last_peered":"2024-04-26T22:27:14.985270+0000","last_clean":"2024-04-26T22:27:14.985270+0000","last_became_active":"2024-04-26T22:27:13.870203+0000","last_became_peered":"2024-04-26T22:27:13.870203+0000","last_unstale":"2024-04-26T22:27:14.985270+0000","last_undegraded":"2024-04-26T22:27:14.985270+0000","last_fullsized":"2024-04-26T22:27:14.985270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879610+0000","last_change":"2024-04-26T22:27:12.866037+0000","last_active":"2024-04-26T22:27:14.879610+0000","last_peered":"2024-04-26T22:27:14.879610+0000","last_clean":"2024-04-26T22:27:14.879610+0000","last_became_active":"2024-04-26T22:27:12.865735+0000","last_became_peered":"2024-04-26T22:27:12.865735+0000","last_unstale":"2024-04-26T22:27:14.879610+0000","last_undegraded":"2024-04-26T22:27:14.879610+0000","last_fullsized":"2024-04-26T22:27:14.879610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985764+0000","last_change":"2024-04-26T22:27:12.870843+0000","last_active":"2024-04-26T22:27:14.985764+0000","last_peered":"2024-04-26T22:27:14.985764+0000","last_clean":"2024-04-26T22:27:14.985764+0000","last_became_active":"2024-04-26T22:27:12.870301+0000","last_became_peered":"2024-04-26T22:27:12.870301+0000","last_unstale":"2024-04-26T22:27:14.985764+0000","last_undegraded":"2024-04-26T22:27:14.985764+0000","last_fullsized":"2024-04-26T22:27:14.985764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:01.836867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985619+0000","last_change":"2024-04-26T22:27:13.871910+0000","last_active":"2024-04-26T22:27:14.985619+0000","last_peered":"2024-04-26T22:27:14.985619+0000","last_clean":"2024-04-26T22:27:14.985619+0000","last_became_active":"2024-04-26T22:27:13.871645+0000","last_became_peered":"2024-04-26T22:27:13.871645+0000","last_unstale":"2024-04-26T22:27:14.985619+0000","last_undegraded":"2024-04-26T22:27:14.985619+0000","last_fullsized":"2024-04-26T22:27:14.985619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.286027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863081+0000","last_change":"2024-04-26T22:27:12.860052+0000","last_active":"2024-04-26T22:27:14.863081+0000","last_peered":"2024-04-26T22:27:14.863081+0000","last_clean":"2024-04-26T22:27:14.863081+0000","last_became_active":"2024-04-26T22:27:12.859367+0000","last_became_peered":"2024-04-26T22:27:12.859367+0000","last_unstale":"2024-04-26T22:27:14.863081+0000","last_undegraded":"2024-04-26T22:27:14.863081+0000","last_fullsized":"2024-04-26T22:27:14.863081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861594+0000","last_change":"2024-04-26T22:27:13.868468+0000","last_active":"2024-04-26T22:27:14.861594+0000","last_peered":"2024-04-26T22:27:14.861594+0000","last_clean":"2024-04-26T22:27:14.861594+0000","last_became_active":"2024-04-26T22:27:13.868172+0000","last_became_peered":"2024-04-26T22:27:13.868172+0000","last_unstale":"2024-04-26T22:27:14.861594+0000","last_undegraded":"2024-04-26T22:27:14.861594+0000","last_fullsized":"2024-04-26T22:27:14.861594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879909+0000","last_change":"2024-04-26T22:27:12.865874+0000","last_active":"2024-04-26T22:27:14.879909+0000","last_peered":"2024-04-26T22:27:14.879909+0000","last_clean":"2024-04-26T22:27:14.879909+0000","last_became_active":"2024-04-26T22:27:12.865593+0000","last_became_peered":"2024-04-26T22:27:12.865593+0000","last_unstale":"2024-04-26T22:27:14.879909+0000","last_undegraded":"2024-04-26T22:27:14.879909+0000","last_fullsized":"2024-04-26T22:27:14.879909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863394+0000","last_change":"2024-04-26T22:27:13.881094+0000","last_active":"2024-04-26T22:27:14.863394+0000","last_peered":"2024-04-26T22:27:14.863394+0000","last_clean":"2024-04-26T22:27:14.863394+0000","last_became_active":"2024-04-26T22:27:13.880021+0000","last_became_peered":"2024-04-26T22:27:13.880021+0000","last_unstale":"2024-04-26T22:27:14.863394+0000","last_undegraded":"2024-04-26T22:27:14.863394+0000","last_fullsized":"2024-04-26T22:27:14.863394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863254+0000","last_change":"2024-04-26T22:27:12.859862+0000","last_active":"2024-04-26T22:27:14.863254+0000","last_peered":"2024-04-26T22:27:14.863254+0000","last_clean":"2024-04-26T22:27:14.863254+0000","last_became_active":"2024-04-26T22:27:12.859130+0000","last_became_peered":"2024-04-26T22:27:12.859130+0000","last_unstale":"2024-04-26T22:27:14.863254+0000","last_undegraded":"2024-04-26T22:27:14.863254+0000","last_fullsized":"2024-04-26T22:27:14.863254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985290+0000","last_change":"2024-04-26T22:27:13.876436+0000","last_active":"2024-04-26T22:27:14.985290+0000","last_peered":"2024-04-26T22:27:14.985290+0000","last_clean":"2024-04-26T22:27:14.985290+0000","last_became_active":"2024-04-26T22:27:13.876123+0000","last_became_peered":"2024-04-26T22:27:13.876123+0000","last_unstale":"2024-04-26T22:27:14.985290+0000","last_undegraded":"2024-04-26T22:27:14.985290+0000","last_fullsized":"2024-04-26T22:27:14.985290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861670+0000","last_change":"2024-04-26T22:27:12.864622+0000","last_active":"2024-04-26T22:27:14.861670+0000","last_peered":"2024-04-26T22:27:14.861670+0000","last_clean":"2024-04-26T22:27:14.861670+0000","last_became_active":"2024-04-26T22:27:12.864324+0000","last_became_peered":"2024-04-26T22:27:12.864324+0000","last_unstale":"2024-04-26T22:27:14.861670+0000","last_undegraded":"2024-04-26T22:27:14.861670+0000","last_fullsized":"2024-04-26T22:27:14.861670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863255+0000","last_change":"2024-04-26T22:27:13.874206+0000","last_active":"2024-04-26T22:27:14.863255+0000","last_peered":"2024-04-26T22:27:14.863255+0000","last_clean":"2024-04-26T22:27:14.863255+0000","last_became_active":"2024-04-26T22:27:13.873944+0000","last_became_peered":"2024-04-26T22:27:13.873944+0000","last_unstale":"2024-04-26T22:27:14.863255+0000","last_undegraded":"2024-04-26T22:27:14.863255+0000","last_fullsized":"2024-04-26T22:27:14.863255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:17.274394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879809+0000","last_change":"2024-04-26T22:27:12.865709+0000","last_active":"2024-04-26T22:27:14.879809+0000","last_peered":"2024-04-26T22:27:14.879809+0000","last_clean":"2024-04-26T22:27:14.879809+0000","last_became_active":"2024-04-26T22:27:12.865398+0000","last_became_peered":"2024-04-26T22:27:12.865398+0000","last_unstale":"2024-04-26T22:27:14.879809+0000","last_undegraded":"2024-04-26T22:27:14.879809+0000","last_fullsized":"2024-04-26T22:27:14.879809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985173+0000","last_change":"2024-04-26T22:27:13.886280+0000","last_active":"2024-04-26T22:27:14.985173+0000","last_peered":"2024-04-26T22:27:14.985173+0000","last_clean":"2024-04-26T22:27:14.985173+0000","last_became_active":"2024-04-26T22:27:13.886020+0000","last_became_peered":"2024-04-26T22:27:13.886020+0000","last_unstale":"2024-04-26T22:27:14.985173+0000","last_undegraded":"2024-04-26T22:27:14.985173+0000","last_fullsized":"2024-04-26T22:27:14.985173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985378+0000","last_change":"2024-04-26T22:27:12.864771+0000","last_active":"2024-04-26T22:27:14.985378+0000","last_peered":"2024-04-26T22:27:14.985378+0000","last_clean":"2024-04-26T22:27:14.985378+0000","last_became_active":"2024-04-26T22:27:12.864215+0000","last_became_peered":"2024-04-26T22:27:12.864215+0000","last_unstale":"2024-04-26T22:27:14.985378+0000","last_undegraded":"2024-04-26T22:27:14.985378+0000","last_fullsized":"2024-04-26T22:27:14.985378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:09.919428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984993+0000","last_change":"2024-04-26T22:27:13.877703+0000","last_active":"2024-04-26T22:27:14.984993+0000","last_peered":"2024-04-26T22:27:14.984993+0000","last_clean":"2024-04-26T22:27:14.984993+0000","last_became_active":"2024-04-26T22:27:13.877406+0000","last_became_peered":"2024-04-26T22:27:13.877406+0000","last_unstale":"2024-04-26T22:27:14.984993+0000","last_undegraded":"2024-04-26T22:27:14.984993+0000","last_fullsized":"2024-04-26T22:27:14.984993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:36.849288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863454+0000","last_change":"2024-04-26T22:27:12.860680+0000","last_active":"2024-04-26T22:27:14.863454+0000","last_peered":"2024-04-26T22:27:14.863454+0000","last_clean":"2024-04-26T22:27:14.863454+0000","last_became_active":"2024-04-26T22:27:12.860342+0000","last_became_peered":"2024-04-26T22:27:12.860342+0000","last_unstale":"2024-04-26T22:27:14.863454+0000","last_undegraded":"2024-04-26T22:27:14.863454+0000","last_fullsized":"2024-04-26T22:27:14.863454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985217+0000","last_change":"2024-04-26T22:27:13.887857+0000","last_active":"2024-04-26T22:27:14.985217+0000","last_peered":"2024-04-26T22:27:14.985217+0000","last_clean":"2024-04-26T22:27:14.985217+0000","last_became_active":"2024-04-26T22:27:13.887578+0000","last_became_peered":"2024-04-26T22:27:13.887578+0000","last_unstale":"2024-04-26T22:27:14.985217+0000","last_undegraded":"2024-04-26T22:27:14.985217+0000","last_fullsized":"2024-04-26T22:27:14.985217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:52:32.399930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879621+0000","last_change":"2024-04-26T22:27:12.863476+0000","last_active":"2024-04-26T22:27:14.879621+0000","last_peered":"2024-04-26T22:27:14.879621+0000","last_clean":"2024-04-26T22:27:14.879621+0000","last_became_active":"2024-04-26T22:27:12.863172+0000","last_became_peered":"2024-04-26T22:27:12.863172+0000","last_unstale":"2024-04-26T22:27:14.879621+0000","last_undegraded":"2024-04-26T22:27:14.879621+0000","last_fullsized":"2024-04-26T22:27:14.879621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879712+0000","last_change":"2024-04-26T22:27:13.885569+0000","last_active":"2024-04-26T22:27:14.879712+0000","last_peered":"2024-04-26T22:27:14.879712+0000","last_clean":"2024-04-26T22:27:14.879712+0000","last_became_active":"2024-04-26T22:27:13.885255+0000","last_became_peered":"2024-04-26T22:27:13.885255+0000","last_unstale":"2024-04-26T22:27:14.879712+0000","last_undegraded":"2024-04-26T22:27:14.879712+0000","last_fullsized":"2024-04-26T22:27:14.879712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:09:36.477106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984474+0000","last_change":"2024-04-26T22:27:12.856942+0000","last_active":"2024-04-26T22:27:14.984474+0000","last_peered":"2024-04-26T22:27:14.984474+0000","last_clean":"2024-04-26T22:27:14.984474+0000","last_became_active":"2024-04-26T22:27:12.856199+0000","last_became_peered":"2024-04-26T22:27:12.856199+0000","last_unstale":"2024-04-26T22:27:14.984474+0000","last_undegraded":"2024-04-26T22:27:14.984474+0000","last_fullsized":"2024-04-26T22:27:14.984474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:11:51.471444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984932+0000","last_change":"2024-04-26T22:27:13.888422+0000","last_active":"2024-04-26T22:27:14.984932+0000","last_peered":"2024-04-26T22:27:14.984932+0000","last_clean":"2024-04-26T22:27:14.984932+0000","last_became_active":"2024-04-26T22:27:13.888063+0000","last_became_peered":"2024-04-26T22:27:13.888063+0000","last_unstale":"2024-04-26T22:27:14.984932+0000","last_undegraded":"2024-04-26T22:27:14.984932+0000","last_fullsized":"2024-04-26T22:27:14.984932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:19:29.841451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863321+0000","last_change":"2024-04-26T22:27:12.863291+0000","last_active":"2024-04-26T22:27:14.863321+0000","last_peered":"2024-04-26T22:27:14.863321+0000","last_clean":"2024-04-26T22:27:14.863321+0000","last_became_active":"2024-04-26T22:27:12.863016+0000","last_became_peered":"2024-04-26T22:27:12.863016+0000","last_unstale":"2024-04-26T22:27:14.863321+0000","last_undegraded":"2024-04-26T22:27:14.863321+0000","last_fullsized":"2024-04-26T22:27:14.863321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985407+0000","last_change":"2024-04-26T22:27:13.880545+0000","last_active":"2024-04-26T22:27:14.985407+0000","last_peered":"2024-04-26T22:27:14.985407+0000","last_clean":"2024-04-26T22:27:14.985407+0000","last_became_active":"2024-04-26T22:27:13.880053+0000","last_became_peered":"2024-04-26T22:27:13.880053+0000","last_unstale":"2024-04-26T22:27:14.985407+0000","last_undegraded":"2024-04-26T22:27:14.985407+0000","last_fullsized":"2024-04-26T22:27:14.985407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984817+0000","last_change":"2024-04-26T22:27:12.856649+0000","last_active":"2024-04-26T22:27:14.984817+0000","last_peered":"2024-04-26T22:27:14.984817+0000","last_clean":"2024-04-26T22:27:14.984817+0000","last_became_active":"2024-04-26T22:27:12.856005+0000","last_became_peered":"2024-04-26T22:27:12.856005+0000","last_unstale":"2024-04-26T22:27:14.984817+0000","last_undegraded":"2024-04-26T22:27:14.984817+0000","last_fullsized":"2024-04-26T22:27:14.984817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985551+0000","last_change":"2024-04-26T22:27:13.888874+0000","last_active":"2024-04-26T22:27:14.985551+0000","last_peered":"2024-04-26T22:27:14.985551+0000","last_clean":"2024-04-26T22:27:14.985551+0000","last_became_active":"2024-04-26T22:27:13.888500+0000","last_became_peered":"2024-04-26T22:27:13.888500+0000","last_unstale":"2024-04-26T22:27:14.985551+0000","last_undegraded":"2024-04-26T22:27:14.985551+0000","last_fullsized":"2024-04-26T22:27:14.985551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:55.006561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861789+0000","last_change":"2024-04-26T22:27:12.863335+0000","last_active":"2024-04-26T22:27:14.861789+0000","last_peered":"2024-04-26T22:27:14.861789+0000","last_clean":"2024-04-26T22:27:14.861789+0000","last_became_active":"2024-04-26T22:27:12.862860+0000","last_became_peered":"2024-04-26T22:27:12.862860+0000","last_unstale":"2024-04-26T22:27:14.861789+0000","last_undegraded":"2024-04-26T22:27:14.861789+0000","last_fullsized":"2024-04-26T22:27:14.861789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864321+0000","last_change":"2024-04-26T22:27:13.882256+0000","last_active":"2024-04-26T22:27:14.864321+0000","last_peered":"2024-04-26T22:27:14.864321+0000","last_clean":"2024-04-26T22:27:14.864321+0000","last_became_active":"2024-04-26T22:27:13.882007+0000","last_became_peered":"2024-04-26T22:27:13.882007+0000","last_unstale":"2024-04-26T22:27:14.864321+0000","last_undegraded":"2024-04-26T22:27:14.864321+0000","last_fullsized":"2024-04-26T22:27:14.864321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:41.527545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984706+0000","last_change":"2024-04-26T22:27:12.866195+0000","last_active":"2024-04-26T22:27:14.984706+0000","last_peered":"2024-04-26T22:27:14.984706+0000","last_clean":"2024-04-26T22:27:14.984706+0000","last_became_active":"2024-04-26T22:27:12.865877+0000","last_became_peered":"2024-04-26T22:27:12.865877+0000","last_unstale":"2024-04-26T22:27:14.984706+0000","last_undegraded":"2024-04-26T22:27:14.984706+0000","last_fullsized":"2024-04-26T22:27:14.984706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985189+0000","last_change":"2024-04-26T22:27:13.887105+0000","last_active":"2024-04-26T22:27:14.985189+0000","last_peered":"2024-04-26T22:27:14.985189+0000","last_clean":"2024-04-26T22:27:14.985189+0000","last_became_active":"2024-04-26T22:27:13.886847+0000","last_became_peered":"2024-04-26T22:27:13.886847+0000","last_unstale":"2024-04-26T22:27:14.985189+0000","last_undegraded":"2024-04-26T22:27:14.985189+0000","last_fullsized":"2024-04-26T22:27:14.985189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:39:51.528409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984901+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:27:14.984901+0000","last_peered":"2024-04-26T22:27:14.984901+0000","last_clean":"2024-04-26T22:27:14.984901+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:27:14.984901+0000","last_undegraded":"2024-04-26T22:27:14.984901+0000","last_fullsized":"2024-04-26T22:27:14.984901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863214+0000","last_change":"2024-04-26T22:27:12.855353+0000","last_active":"2024-04-26T22:27:14.863214+0000","last_peered":"2024-04-26T22:27:14.863214+0000","last_clean":"2024-04-26T22:27:14.863214+0000","last_became_active":"2024-04-26T22:27:12.854814+0000","last_became_peered":"2024-04-26T22:27:12.854814+0000","last_unstale":"2024-04-26T22:27:14.863214+0000","last_undegraded":"2024-04-26T22:27:14.863214+0000","last_fullsized":"2024-04-26T22:27:14.863214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985768+0000","last_change":"2024-04-26T22:27:13.888534+0000","last_active":"2024-04-26T22:27:14.985768+0000","last_peered":"2024-04-26T22:27:14.985768+0000","last_clean":"2024-04-26T22:27:14.985768+0000","last_became_active":"2024-04-26T22:27:13.888091+0000","last_became_peered":"2024-04-26T22:27:13.888091+0000","last_unstale":"2024-04-26T22:27:14.985768+0000","last_undegraded":"2024-04-26T22:27:14.985768+0000","last_fullsized":"2024-04-26T22:27:14.985768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984835+0000","last_change":"2024-04-26T22:27:12.869665+0000","last_active":"2024-04-26T22:27:14.984835+0000","last_peered":"2024-04-26T22:27:14.984835+0000","last_clean":"2024-04-26T22:27:14.984835+0000","last_became_active":"2024-04-26T22:27:12.869352+0000","last_became_peered":"2024-04-26T22:27:12.869352+0000","last_unstale":"2024-04-26T22:27:14.984835+0000","last_undegraded":"2024-04-26T22:27:14.984835+0000","last_fullsized":"2024-04-26T22:27:14.984835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:35.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985026+0000","last_change":"2024-04-26T22:27:13.869644+0000","last_active":"2024-04-26T22:27:14.985026+0000","last_peered":"2024-04-26T22:27:14.985026+0000","last_clean":"2024-04-26T22:27:14.985026+0000","last_became_active":"2024-04-26T22:27:13.869375+0000","last_became_peered":"2024-04-26T22:27:13.869375+0000","last_unstale":"2024-04-26T22:27:14.985026+0000","last_undegraded":"2024-04-26T22:27:14.985026+0000","last_fullsized":"2024-04-26T22:27:14.985026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:39.482073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984581+0000","last_change":"2024-04-26T22:27:12.859680+0000","last_active":"2024-04-26T22:27:14.984581+0000","last_peered":"2024-04-26T22:27:14.984581+0000","last_clean":"2024-04-26T22:27:14.984581+0000","last_became_active":"2024-04-26T22:27:12.857700+0000","last_became_peered":"2024-04-26T22:27:12.857700+0000","last_unstale":"2024-04-26T22:27:14.984581+0000","last_undegraded":"2024-04-26T22:27:14.984581+0000","last_fullsized":"2024-04-26T22:27:14.984581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861737+0000","last_change":"2024-04-26T22:27:13.870073+0000","last_active":"2024-04-26T22:27:14.861737+0000","last_peered":"2024-04-26T22:27:14.861737+0000","last_clean":"2024-04-26T22:27:14.861737+0000","last_became_active":"2024-04-26T22:27:13.869803+0000","last_became_peered":"2024-04-26T22:27:13.869803+0000","last_unstale":"2024-04-26T22:27:14.861737+0000","last_undegraded":"2024-04-26T22:27:14.861737+0000","last_fullsized":"2024-04-26T22:27:14.861737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863550+0000","last_change":"2024-04-26T22:27:12.859881+0000","last_active":"2024-04-26T22:27:14.863550+0000","last_peered":"2024-04-26T22:27:14.863550+0000","last_clean":"2024-04-26T22:27:14.863550+0000","last_became_active":"2024-04-26T22:27:12.859599+0000","last_became_peered":"2024-04-26T22:27:12.859599+0000","last_unstale":"2024-04-26T22:27:14.863550+0000","last_undegraded":"2024-04-26T22:27:14.863550+0000","last_fullsized":"2024-04-26T22:27:14.863550+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863194+0000","last_change":"2024-04-26T22:27:13.873492+0000","last_active":"2024-04-26T22:27:14.863194+0000","last_peered":"2024-04-26T22:27:14.863194+0000","last_clean":"2024-04-26T22:27:14.863194+0000","last_became_active":"2024-04-26T22:27:13.873235+0000","last_became_peered":"2024-04-26T22:27:13.873235+0000","last_unstale":"2024-04-26T22:27:14.863194+0000","last_undegraded":"2024-04-26T22:27:14.863194+0000","last_fullsized":"2024-04-26T22:27:14.863194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:57.982466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861708+0000","last_change":"2024-04-26T22:27:12.863689+0000","last_active":"2024-04-26T22:27:14.861708+0000","last_peered":"2024-04-26T22:27:14.861708+0000","last_clean":"2024-04-26T22:27:14.861708+0000","last_became_active":"2024-04-26T22:27:12.862689+0000","last_became_peered":"2024-04-26T22:27:12.862689+0000","last_unstale":"2024-04-26T22:27:14.861708+0000","last_undegraded":"2024-04-26T22:27:14.861708+0000","last_fullsized":"2024-04-26T22:27:14.861708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863992+0000","last_change":"2024-04-26T22:27:13.882989+0000","last_active":"2024-04-26T22:27:14.863992+0000","last_peered":"2024-04-26T22:27:14.863992+0000","last_clean":"2024-04-26T22:27:14.863992+0000","last_became_active":"2024-04-26T22:27:13.882731+0000","last_became_peered":"2024-04-26T22:27:13.882731+0000","last_unstale":"2024-04-26T22:27:14.863992+0000","last_undegraded":"2024-04-26T22:27:14.863992+0000","last_fullsized":"2024-04-26T22:27:14.863992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:50.971156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985421+0000","last_change":"2024-04-26T22:27:12.869598+0000","last_active":"2024-04-26T22:27:14.985421+0000","last_peered":"2024-04-26T22:27:14.985421+0000","last_clean":"2024-04-26T22:27:14.985421+0000","last_became_active":"2024-04-26T22:27:12.869301+0000","last_became_peered":"2024-04-26T22:27:12.869301+0000","last_unstale":"2024-04-26T22:27:14.985421+0000","last_undegraded":"2024-04-26T22:27:14.985421+0000","last_fullsized":"2024-04-26T22:27:14.985421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:03:33.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861610+0000","last_change":"2024-04-26T22:27:13.868115+0000","last_active":"2024-04-26T22:27:14.861610+0000","last_peered":"2024-04-26T22:27:14.861610+0000","last_clean":"2024-04-26T22:27:14.861610+0000","last_became_active":"2024-04-26T22:27:13.867854+0000","last_became_peered":"2024-04-26T22:27:13.867854+0000","last_unstale":"2024-04-26T22:27:14.861610+0000","last_undegraded":"2024-04-26T22:27:14.861610+0000","last_fullsized":"2024-04-26T22:27:14.861610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879222+0000","last_change":"2024-04-26T22:27:12.867931+0000","last_active":"2024-04-26T22:27:14.879222+0000","last_peered":"2024-04-26T22:27:14.879222+0000","last_clean":"2024-04-26T22:27:14.879222+0000","last_became_active":"2024-04-26T22:27:12.867588+0000","last_became_peered":"2024-04-26T22:27:12.867588+0000","last_unstale":"2024-04-26T22:27:14.879222+0000","last_undegraded":"2024-04-26T22:27:14.879222+0000","last_fullsized":"2024-04-26T22:27:14.879222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986195+0000","last_change":"2024-04-26T22:27:13.877905+0000","last_active":"2024-04-26T22:27:14.986195+0000","last_peered":"2024-04-26T22:27:14.986195+0000","last_clean":"2024-04-26T22:27:14.986195+0000","last_became_active":"2024-04-26T22:27:13.877626+0000","last_became_peered":"2024-04-26T22:27:13.877626+0000","last_unstale":"2024-04-26T22:27:14.986195+0000","last_undegraded":"2024-04-26T22:27:14.986195+0000","last_fullsized":"2024-04-26T22:27:14.986195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879942+0000","last_change":"2024-04-26T22:27:12.866913+0000","last_active":"2024-04-26T22:27:14.879942+0000","last_peered":"2024-04-26T22:27:14.879942+0000","last_clean":"2024-04-26T22:27:14.879942+0000","last_became_active":"2024-04-26T22:27:12.866428+0000","last_became_peered":"2024-04-26T22:27:12.866428+0000","last_unstale":"2024-04-26T22:27:14.879942+0000","last_undegraded":"2024-04-26T22:27:14.879942+0000","last_fullsized":"2024-04-26T22:27:14.879942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:08:56.976940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985432+0000","last_change":"2024-04-26T22:27:13.867975+0000","last_active":"2024-04-26T22:27:14.985432+0000","last_peered":"2024-04-26T22:27:14.985432+0000","last_clean":"2024-04-26T22:27:14.985432+0000","last_became_active":"2024-04-26T22:27:13.867657+0000","last_became_peered":"2024-04-26T22:27:13.867657+0000","last_unstale":"2024-04-26T22:27:14.985432+0000","last_undegraded":"2024-04-26T22:27:14.985432+0000","last_fullsized":"2024-04-26T22:27:14.985432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:59.016618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.984833+0000","last_change":"2024-04-26T22:27:12.870598+0000","last_active":"2024-04-26T22:27:14.984833+0000","last_peered":"2024-04-26T22:27:14.984833+0000","last_clean":"2024-04-26T22:27:14.984833+0000","last_became_active":"2024-04-26T22:27:12.870147+0000","last_became_peered":"2024-04-26T22:27:12.870147+0000","last_unstale":"2024-04-26T22:27:14.984833+0000","last_undegraded":"2024-04-26T22:27:14.984833+0000","last_fullsized":"2024-04-26T22:27:14.984833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:16.268868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863751+0000","last_change":"2024-04-26T22:27:13.869914+0000","last_active":"2024-04-26T22:27:14.863751+0000","last_peered":"2024-04-26T22:27:14.863751+0000","last_clean":"2024-04-26T22:27:14.863751+0000","last_became_active":"2024-04-26T22:27:13.869626+0000","last_became_peered":"2024-04-26T22:27:13.869626+0000","last_unstale":"2024-04-26T22:27:14.863751+0000","last_undegraded":"2024-04-26T22:27:14.863751+0000","last_fullsized":"2024-04-26T22:27:14.863751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:13.509088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861709+0000","last_change":"2024-04-26T22:27:12.865574+0000","last_active":"2024-04-26T22:27:14.861709+0000","last_peered":"2024-04-26T22:27:14.861709+0000","last_clean":"2024-04-26T22:27:14.861709+0000","last_became_active":"2024-04-26T22:27:12.865293+0000","last_became_peered":"2024-04-26T22:27:12.865293+0000","last_unstale":"2024-04-26T22:27:14.861709+0000","last_undegraded":"2024-04-26T22:27:14.861709+0000","last_fullsized":"2024-04-26T22:27:14.861709+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:50:25.316768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985793+0000","last_change":"2024-04-26T22:27:13.873495+0000","last_active":"2024-04-26T22:27:14.985793+0000","last_peered":"2024-04-26T22:27:14.985793+0000","last_clean":"2024-04-26T22:27:14.985793+0000","last_became_active":"2024-04-26T22:27:13.873110+0000","last_became_peered":"2024-04-26T22:27:13.873110+0000","last_unstale":"2024-04-26T22:27:14.985793+0000","last_undegraded":"2024-04-26T22:27:14.985793+0000","last_fullsized":"2024-04-26T22:27:14.985793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.861881+0000","last_change":"2024-04-26T22:27:12.860253+0000","last_active":"2024-04-26T22:27:14.861881+0000","last_peered":"2024-04-26T22:27:14.861881+0000","last_clean":"2024-04-26T22:27:14.861881+0000","last_became_active":"2024-04-26T22:27:12.859755+0000","last_became_peered":"2024-04-26T22:27:12.859755+0000","last_unstale":"2024-04-26T22:27:14.861881+0000","last_undegraded":"2024-04-26T22:27:14.861881+0000","last_fullsized":"2024-04-26T22:27:14.861881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:15:21.431453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986034+0000","last_change":"2024-04-26T22:27:13.875140+0000","last_active":"2024-04-26T22:27:14.986034+0000","last_peered":"2024-04-26T22:27:14.986034+0000","last_clean":"2024-04-26T22:27:14.986034+0000","last_became_active":"2024-04-26T22:27:13.872838+0000","last_became_peered":"2024-04-26T22:27:13.872838+0000","last_unstale":"2024-04-26T22:27:14.986034+0000","last_undegraded":"2024-04-26T22:27:14.986034+0000","last_fullsized":"2024-04-26T22:27:14.986034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:31.392715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985243+0000","last_change":"2024-04-26T22:27:12.867920+0000","last_active":"2024-04-26T22:27:14.985243+0000","last_peered":"2024-04-26T22:27:14.985243+0000","last_clean":"2024-04-26T22:27:14.985243+0000","last_became_active":"2024-04-26T22:27:12.867652+0000","last_became_peered":"2024-04-26T22:27:12.867652+0000","last_unstale":"2024-04-26T22:27:14.985243+0000","last_undegraded":"2024-04-26T22:27:14.985243+0000","last_fullsized":"2024-04-26T22:27:14.985243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:36.571464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.879782+0000","last_change":"2024-04-26T22:27:13.885168+0000","last_active":"2024-04-26T22:27:14.879782+0000","last_peered":"2024-04-26T22:27:14.879782+0000","last_clean":"2024-04-26T22:27:14.879782+0000","last_became_active":"2024-04-26T22:27:13.884882+0000","last_became_peered":"2024-04-26T22:27:13.884882+0000","last_unstale":"2024-04-26T22:27:14.879782+0000","last_undegraded":"2024-04-26T22:27:14.879782+0000","last_fullsized":"2024-04-26T22:27:14.879782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:14.353394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.863833+0000","last_change":"2024-04-26T22:27:13.878648+0000","last_active":"2024-04-26T22:27:14.863833+0000","last_peered":"2024-04-26T22:27:14.863833+0000","last_clean":"2024-04-26T22:27:14.863833+0000","last_became_active":"2024-04-26T22:27:13.878280+0000","last_became_peered":"2024-04-26T22:27:13.878280+0000","last_unstale":"2024-04-26T22:27:14.863833+0000","last_undegraded":"2024-04-26T22:27:14.863833+0000","last_fullsized":"2024-04-26T22:27:14.863833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:54.014620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985097+0000","last_change":"2024-04-26T22:27:12.865334+0000","last_active":"2024-04-26T22:27:14.985097+0000","last_peered":"2024-04-26T22:27:14.985097+0000","last_clean":"2024-04-26T22:27:14.985097+0000","last_became_active":"2024-04-26T22:27:12.865055+0000","last_became_peered":"2024-04-26T22:27:12.865055+0000","last_unstale":"2024-04-26T22:27:14.985097+0000","last_undegraded":"2024-04-26T22:27:14.985097+0000","last_fullsized":"2024-04-26T22:27:14.985097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:12.503562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.864217+0000","last_change":"2024-04-26T22:27:13.879604+0000","last_active":"2024-04-26T22:27:14.864217+0000","last_peered":"2024-04-26T22:27:14.864217+0000","last_clean":"2024-04-26T22:27:14.864217+0000","last_became_active":"2024-04-26T22:27:13.879317+0000","last_became_peered":"2024-04-26T22:27:13.879317+0000","last_unstale":"2024-04-26T22:27:14.864217+0000","last_undegraded":"2024-04-26T22:27:14.864217+0000","last_fullsized":"2024-04-26T22:27:14.864217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:27:49.052832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985330+0000","last_change":"2024-04-26T22:27:12.870741+0000","last_active":"2024-04-26T22:27:14.985330+0000","last_peered":"2024-04-26T22:27:14.985330+0000","last_clean":"2024-04-26T22:27:14.985330+0000","last_became_active":"2024-04-26T22:27:12.870356+0000","last_became_peered":"2024-04-26T22:27:12.870356+0000","last_unstale":"2024-04-26T22:27:14.985330+0000","last_undegraded":"2024-04-26T22:27:14.985330+0000","last_fullsized":"2024-04-26T22:27:14.985330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:38.747576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:13.888713+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:13.888393+0000","last_became_peered":"2024-04-26T22:27:13.888393+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:21.362561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.986139+0000","last_change":"2024-04-26T22:27:12.870271+0000","last_active":"2024-04-26T22:27:14.986139+0000","last_peered":"2024-04-26T22:27:14.986139+0000","last_clean":"2024-04-26T22:27:14.986139+0000","last_became_active":"2024-04-26T22:27:12.869919+0000","last_became_peered":"2024-04-26T22:27:12.869919+0000","last_unstale":"2024-04-26T22:27:14.986139+0000","last_undegraded":"2024-04-26T22:27:14.986139+0000","last_fullsized":"2024-04-26T22:27:14.986139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:15:33.268650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985725+0000","last_change":"2024-04-26T22:27:13.883151+0000","last_active":"2024-04-26T22:27:14.985725+0000","last_peered":"2024-04-26T22:27:14.985725+0000","last_clean":"2024-04-26T22:27:14.985725+0000","last_became_active":"2024-04-26T22:27:13.882875+0000","last_became_peered":"2024-04-26T22:27:13.882875+0000","last_unstale":"2024-04-26T22:27:14.985725+0000","last_undegraded":"2024-04-26T22:27:14.985725+0000","last_fullsized":"2024-04-26T22:27:14.985725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:12.843258+0000","last_clean_scrub_stamp":"2024-04-26T22:27:12.843258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:34.010090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-26T22:27:14.985454+0000","last_change":"2024-04-26T22:27:12.865998+0000","last_active":"2024-04-26T22:27:14.985454+0000","last_peered":"2024-04-26T22:27:14.985454+0000","last_clean":"2024-04-26T22:27:14.985454+0000","last_became_active":"2024-04-26T22:27:12.865683+0000","last_became_peered":"2024-04-26T22:27:12.865683+0000","last_unstale":"2024-04-26T22:27:14.985454+0000","last_undegraded":"2024-04-26T22:27:14.985454+0000","last_fullsized":"2024-04-26T22:27:14.985454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:11.837732+0000","last_clean_scrub_stamp":"2024-04-26T22:27:11.837732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:41.634062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:27:20.980 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T22:27:20.980 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T22:27:20.980 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T22:27:20.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T22:27:21.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.115+0000 7fa748cd1640 1 Processor -- start 2024-04-26T22:27:21.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.115+0000 7fa748cd1640 1 -- start start 2024-04-26T22:27:21.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa748cd1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa7440d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa748cd1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa7440ccd00 0x7fa744073080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa748cd1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7440cd1c0 con 0x7fa7440d0e70 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa748cd1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa7440cddd0 con 0x7fa7440ccd00 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa748cd1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7440cd4a0 con 0x7fa74415c330 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa742d9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa7440d0e70 0x7fa74415fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56478/0 (socket says 172.21.15.62:56478) 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.116+0000 7fa742d9d640 1 -- 172.21.15.62:0/1004287632 learned_addr learned my addr 172.21.15.62:0/1004287632 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa74259c640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa7440d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa73bfff640 1 --2- 172.21.15.62:0/1004287632 >> v2:172.21.15.174:3300/0 conn(0x7fa7440ccd00 0x7fa744073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2075835870 0 0) 0x7fa7440cd1c0 con 0x7fa7440d0e70 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa730005730 con 0x7fa7440d0e70 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 578445928 0 0) 0x7fa730005730 con 0x7fa7440d0e70 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 msgr2=0x7fa7440d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa7440d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 >> v2:172.21.15.174:3300/0 conn(0x7fa7440ccd00 msgr2=0x7fa744073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 --2- 172.21.15.62:0/1004287632 >> v2:172.21.15.174:3300/0 conn(0x7fa7440ccd00 0x7fa744073080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7440cd780 con 0x7fa7440d0e70 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fa73c0019f0 con 0x7fa7440d0e70 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 >> v1:172.21.15.62:6789/0 conn(0x7fa7440d0e70 legacy=0x7fa74415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa730005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa730006140 0x7fa730006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa730006f50 0x7fa7300073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440cddd0 con 0x7fa730006f50 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440cd4a0 con 0x7fa730006140 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa73000dbd0 con 0x7fa74415c330 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa730006f50 0x7fa7300073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa74259c640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa730005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa73bfff640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa730006140 0x7fa730006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 -- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 msgr2=0x7fa730005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa74415c330 0x7fa730005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 -- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa730006140 msgr2=0x7fa730006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa730006140 0x7fa730006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.117+0000 7fa742d9d640 1 -- 172.21.15.62:0/1004287632 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa730005730 con 0x7fa730006f50 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.118+0000 7fa742d9d640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa730006f50 0x7fa7300073d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0013d0 tx=0x7fa730007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25b9db87035909 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.118+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00a4e0 con 0x7fa730006f50 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.118+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa73c02f070 con 0x7fa730006f50 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.118+0000 7fa741d9b640 1 -- 172.21.15.62:0/1004287632 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c02bcd0 con 0x7fa730006f50 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 -- 172.21.15.62:0/1004287632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa730006f50 msgr2=0x7fa7300073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 --2- 172.21.15.62:0/1004287632 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa730006f50 0x7fa7300073d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0013d0 tx=0x7fa730007a40 comp rx=0 tx=0).stop 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa742d9d640 1 -- 172.21.15.62:0/1004287632 reap_dead start 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 -- 172.21.15.62:0/1004287632 shutdown_connections 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 -- 172.21.15.62:0/1004287632 >> 172.21.15.62:0/1004287632 conn(0x7fa7440d1e80 msgr2=0x7fa7440d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 -- 172.21.15.62:0/1004287632 shutdown_connections 2024-04-26T22:27:21.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.119+0000 7fa748cd1640 1 -- 172.21.15.62:0/1004287632 wait complete. 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.120+0000 7fa748cd1640 1 Processor -- start 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.120+0000 7fa748cd1640 1 -- start start 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa744156360 0x7fa7440b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7440cba10 0x7fa744156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440ce3e0 con 0x7fa7440b5f10 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7440ce260 con 0x7fa7440cba10 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa748cd1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7440ce560 con 0x7fa744156360 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35678/0 (socket says 172.21.15.62:35678) 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 -- 172.21.15.62:0/4243041908 learned_addr learned my addr 172.21.15.62:0/4243041908 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa74259c640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa744156360 0x7fa7440b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa742d9d640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7440cba10 0x7fa744156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 -- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa744156360 msgr2=0x7fa7440b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa744156360 0x7fa7440b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 -- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7440cba10 msgr2=0x7fa744156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7440cba10 0x7fa744156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.121+0000 7fa73bfff640 1 -- 172.21.15.62:0/4243041908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7440b6590 con 0x7fa7440b5f10 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa73bfff640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa72c011960 tx=0x7fa72c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00faf0 con 0x7fa7440b5f10 2024-04-26T22:27:21.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c00fc90 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7440b6870 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c014440 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.122+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7440b6e30 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.123+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7440693e0 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.123+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa72c012040 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.124+0000 7fa73affd640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 0x7fa72406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.124+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa72c08c330 con 0x7fa7440b5f10 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.124+0000 7fa74259c640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 0x7fa72406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.125+0000 7fa74259c640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 0x7fa72406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7340099e0 tx=0x7fa734008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.128+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa72c019030 con 0x7fa7440b5f10 2024-04-26T22:27:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.354+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa7440cbdf0 con 0x7fa7440b5f10 2024-04-26T22:27:21.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.354+0000 7fa73affd640 1 -- 172.21.15.62:0/4243041908 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa72c05a6d0 con 0x7fa7440b5f10 2024-04-26T22:27:21.354 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:21.355 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T22:27:21.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 msgr2=0x7fa72406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 0x7fa72406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7340099e0 tx=0x7fa734008040 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 msgr2=0x7fa7440cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa72c011960 tx=0x7fa72c011e20 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 shutdown_connections 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa72406c920 0x7fa72406ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7440cba10 0x7fa744156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa7440b5f10 0x7fa7440cb4d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 --2- 172.21.15.62:0/4243041908 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa744156360 0x7fa7440b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 >> 172.21.15.62:0/4243041908 conn(0x7fa7440d1e80 msgr2=0x7fa744073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 shutdown_connections 2024-04-26T22:27:21.358 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.356+0000 7fa748cd1640 1 -- 172.21.15.62:0/4243041908 wait complete. 2024-04-26T22:27:21.368 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T22:27:21.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:21.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.468+0000 7fa8c03af640 1 Processor -- start 2024-04-26T22:27:21.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.468+0000 7fa8c03af640 1 -- start start 2024-04-26T22:27:21.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8c03af640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8b806c240 0x7fa8b806fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8c03af640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b80b59b0 con 0x7fa8b815f4f0 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8c03af640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b8070350 con 0x7fa8b80b1480 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8c03af640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b80b5c70 con 0x7fa8b806c240 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8be14b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8b806c240 0x7fa8b806fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8bd94a640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa8b815f4f0 0x7fa8b806c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56492/0 (socket says 172.21.15.62:56492) 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8be14b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa8b806c240 0x7fa8b806fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35684/0 (socket says 172.21.15.62:35684) 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8bd94a640 1 -- 172.21.15.62:0/2467085692 learned_addr learned my addr 172.21.15.62:0/2467085692 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8be14b640 1 -- 172.21.15.62:0/2467085692 >> v1:172.21.15.174:6789/0 conn(0x7fa8b80b1480 legacy=0x7fa8b80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8be14b640 1 -- 172.21.15.62:0/2467085692 >> v1:172.21.15.62:6789/0 conn(0x7fa8b815f4f0 legacy=0x7fa8b806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.469+0000 7fa8be14b640 1 -- 172.21.15.62:0/2467085692 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b80b5f50 con 0x7fa8b806c240 2024-04-26T22:27:21.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.470+0000 7fa8be14b640 1 --2- 172.21.15.62:0/2467085692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806c240 0x7fa8b806fd70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009d90 tx=0x7fa8a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea0ceb5644571b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.470+0000 7fa8bd149640 1 -- 172.21.15.62:0/2467085692 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8007740 con 0x7fa8b806c240 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.470+0000 7fa8bd149640 1 -- 172.21.15.62:0/2467085692 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a80078e0 con 0x7fa8b806c240 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.470+0000 7fa8bd149640 1 -- 172.21.15.62:0/2467085692 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a8007cb0 con 0x7fa8b806c240 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 -- 172.21.15.62:0/2467085692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806c240 msgr2=0x7fa8b806fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 --2- 172.21.15.62:0/2467085692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806c240 0x7fa8b806fd70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009d90 tx=0x7fa8a802a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 -- 172.21.15.62:0/2467085692 shutdown_connections 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 --2- 172.21.15.62:0/2467085692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806c240 0x7fa8b806fd70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 -- 172.21.15.62:0/2467085692 >> 172.21.15.62:0/2467085692 conn(0x7fa8b80b3b70 msgr2=0x7fa8b80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 -- 172.21.15.62:0/2467085692 shutdown_connections 2024-04-26T22:27:21.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.471+0000 7fa8c03af640 1 -- 172.21.15.62:0/2467085692 wait complete. 2024-04-26T22:27:21.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.472+0000 7fa8c03af640 1 Processor -- start 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.472+0000 7fa8c03af640 1 -- start start 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8b806c240 0x7fa8b8158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8b80b1480 0x7fa8b8151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80b6750 con 0x7fa8b815f4f0 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80b0fc0 con 0x7fa8b806c240 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8c03af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b80b68d0 con 0x7fa8b80b1480 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35686/0 (socket says 172.21.15.62:35686) 2024-04-26T22:27:21.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 -- 172.21.15.62:0/4122310788 learned_addr learned my addr 172.21.15.62:0/4122310788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be14b640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8b806c240 0x7fa8b8158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8bd94a640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8b80b1480 0x7fa8b8151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 -- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8b80b1480 msgr2=0x7fa8b8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8b80b1480 0x7fa8b8151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 -- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8b806c240 msgr2=0x7fa8b8158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8b806c240 0x7fa8b8158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 -- 172.21.15.62:0/4122310788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b8157040 con 0x7fa8b815f4f0 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.473+0000 7fa8be94c640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c3e0 tx=0x7fa8b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.474+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0011790 con 0x7fa8b815f4f0 2024-04-26T22:27:21.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.474+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b0017070 con 0x7fa8b815f4f0 2024-04-26T22:27:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.474+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b81572e0 con 0x7fa8b815f4f0 2024-04-26T22:27:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.474+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0011b70 con 0x7fa8b815f4f0 2024-04-26T22:27:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.474+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b81578a0 con 0x7fa8b815f4f0 2024-04-26T22:27:21.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.475+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b806a360 con 0x7fa8b815f4f0 2024-04-26T22:27:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.475+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8b0011d10 con 0x7fa8b815f4f0 2024-04-26T22:27:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.475+0000 7fa8b6ffd640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 0x7fa8a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.476+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8b008b900 con 0x7fa8b815f4f0 2024-04-26T22:27:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.476+0000 7fa8be14b640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 0x7fa8a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.476+0000 7fa8be14b640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 0x7fa8a406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8034000 tx=0x7fa8a802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.480 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.480+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8b0059d40 con 0x7fa8b815f4f0 2024-04-26T22:27:21.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.665+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8b80b36b0 con 0x7fa8b815f4f0 2024-04-26T22:27:21.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.666+0000 7fa8b6ffd640 1 -- 172.21.15.62:0/4122310788 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa8b0071e20 con 0x7fa8b815f4f0 2024-04-26T22:27:21.667 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:21.667 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 msgr2=0x7fa8a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 0x7fa8a406edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8034000 tx=0x7fa8a802f000 comp rx=0 tx=0).stop 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 msgr2=0x7fa8b815e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c3e0 tx=0x7fa8b000c8a0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 shutdown_connections 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa8a406c8f0 0x7fa8a406edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b815f4f0 0x7fa8b815e390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8b80b1480 0x7fa8b8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 --2- 172.21.15.62:0/4122310788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8b806c240 0x7fa8b8158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 >> 172.21.15.62:0/4122310788 conn(0x7fa8b80b3b70 msgr2=0x7fa8b8155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 shutdown_connections 2024-04-26T22:27:21.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.668+0000 7fa8c03af640 1 -- 172.21.15.62:0/4122310788 wait complete. 2024-04-26T22:27:21.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:21.680 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:21.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.783+0000 7fa56497f640 1 Processor -- start 2024-04-26T22:27:21.784 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.783+0000 7fa56497f640 1 -- start start 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa56497f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa5600d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa56497f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5600cd640 con 0x7fa5600cff80 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa56497f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5600cd920 con 0x7fa5600b2290 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa56497f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5600cdc00 con 0x7fa5600b2930 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa55dd9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa5600cff80 0x7fa5600d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56508/0 (socket says 172.21.15.62:56508) 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa55dd9b640 1 -- 172.21.15.62:0/3645062385 learned_addr learned my addr 172.21.15.62:0/3645062385 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa55ed9d640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa5600d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708746787 0 0) 0x7fa5600cd640 con 0x7fa5600cff80 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.784+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa54c003520 con 0x7fa5600cff80 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1038394942 0 0) 0x7fa54c003520 con 0x7fa5600cff80 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 msgr2=0x7fa5600d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa5600d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 >> v1:172.21.15.174:6789/0 conn(0x7fa5600b2290 legacy=0x7fa5600cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5600cdee0 con 0x7fa5600cff80 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fa548001670 con 0x7fa5600cff80 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 >> v1:172.21.15.62:6789/0 conn(0x7fa5600cff80 legacy=0x7fa5600d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c0039e0 0x7fa54c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa54c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa54c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5600cdc00 con 0x7fa54c004390 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c00b9c0 con 0x7fa54c0039e0 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa54c00bbc0 con 0x7fa5600b2930 2024-04-26T22:27:21.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa54c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55e59c640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c0039e0 0x7fa54c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55ed9d640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa54c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 -- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 msgr2=0x7fa54c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600b2930 0x7fa54c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 -- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c0039e0 msgr2=0x7fa54c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa54c0039e0 0x7fa54c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.785+0000 7fa55dd9b640 1 -- 172.21.15.62:0/3645062385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c003520 con 0x7fa54c004390 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa54c004860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c190 tx=0x7fa54c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0c584676bf6064 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548007cc0 con 0x7fa54c004390 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa548005ad0 con 0x7fa54c004390 2024-04-26T22:27:21.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa55d59a640 1 -- 172.21.15.62:0/3645062385 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548007070 con 0x7fa54c004390 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa56497f640 1 -- 172.21.15.62:0/3645062385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 msgr2=0x7fa54c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa56497f640 1 --2- 172.21.15.62:0/3645062385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa54c004860 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c190 tx=0x7fa54c005830 comp rx=0 tx=0).stop 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.786+0000 7fa55ed9d640 1 -- 172.21.15.62:0/3645062385 reap_dead start 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.787+0000 7fa56497f640 1 -- 172.21.15.62:0/3645062385 shutdown_connections 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.787+0000 7fa56497f640 1 -- 172.21.15.62:0/3645062385 >> 172.21.15.62:0/3645062385 conn(0x7fa5600d2ff0 msgr2=0x7fa5600d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.787+0000 7fa56497f640 1 -- 172.21.15.62:0/3645062385 shutdown_connections 2024-04-26T22:27:21.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.787+0000 7fa56497f640 1 -- 172.21.15.62:0/3645062385 wait complete. 2024-04-26T22:27:21.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.788+0000 7fa56497f640 1 Processor -- start 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.788+0000 7fa56497f640 1 -- start start 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 0x7fa5600c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600c6a50 0x7fa5600c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5600d26f0 con 0x7fa54c004390 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5600b2670 con 0x7fa5600c8050 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5600d2870 con 0x7fa5600c6a50 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35716/0 (socket says 172.21.15.62:35716) 2024-04-26T22:27:21.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 -- 172.21.15.62:0/3141110786 learned_addr learned my addr 172.21.15.62:0/3141110786 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 0x7fa5600c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55ed9d640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600c6a50 0x7fa5600c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 -- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600c6a50 msgr2=0x7fa5600c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600c6a50 0x7fa5600c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 -- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 msgr2=0x7fa5600c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 0x7fa5600c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 -- 172.21.15.62:0/3141110786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5600c7630 con 0x7fa54c004390 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 0x7fa5600c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa55e59c640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa55800f7c0 tx=0x7fa55800fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55800fe70 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa558018070 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5600c7910 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55800d6d0 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.789+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601572c0 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.790+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56006a360 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.791+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa55800d870 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.791+0000 7fa556ffd640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 0x7fa54006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.791+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa55808b9a0 con 0x7fa54c004390 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.792+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 0x7fa54006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:21.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.792+0000 7fa55dd9b640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 0x7fa54006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c190 tx=0x7fa54802d000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:21.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.796+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa558059e60 con 0x7fa54c004390 2024-04-26T22:27:21.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.984+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5600c7ae0 con 0x7fa54c004390 2024-04-26T22:27:21.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.984+0000 7fa556ffd640 1 -- 172.21.15.62:0/3141110786 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa558059760 con 0x7fa54c004390 2024-04-26T22:27:21.985 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 msgr2=0x7fa54006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 0x7fa54006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c190 tx=0x7fa54802d000 comp rx=0 tx=0).stop 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 msgr2=0x7fa5601f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa55800f7c0 tx=0x7fa55800fc80 comp rx=0 tx=0).stop 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 shutdown_connections 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa54006c920 0x7fa54006ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5600c6a50 0x7fa5600c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5600c8050 0x7fa5600c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.987 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 --2- 172.21.15.62:0/3141110786 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa54c004390 0x7fa5601f4210 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:21.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 >> 172.21.15.62:0/3141110786 conn(0x7fa5600d2ff0 msgr2=0x7fa5600b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:21.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 shutdown_connections 2024-04-26T22:27:21.988 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:21.987+0000 7fa56497f640 1 -- 172.21.15.62:0/3141110786 wait complete. 2024-04-26T22:27:21.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:22.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.101+0000 7f30c9be9640 1 Processor -- start 2024-04-26T22:27:22.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.101+0000 7f30c9be9640 1 -- start start 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c9be9640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30c40b5510 0x7f30c40d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c9be9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30c40b6820 0x7f30c40d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c9be9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30c40c9cc0 con 0x7f30c40b58f0 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c9be9640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f30c40b5d80 con 0x7f30c40b6820 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c9be9640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f30c40b5fd0 con 0x7f30c40b5510 2024-04-26T22:27:22.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c2ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f30c40b6820 0x7f30c40d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f30c40b58f0 0x7f30c40d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56520/0 (socket says 172.21.15.62:56520) 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.102+0000 7f30c3fff640 1 -- 172.21.15.62:0/503708329 learned_addr learned my addr 172.21.15.62:0/503708329 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c37fe640 1 --2- 172.21.15.62:0/503708329 >> v2:172.21.15.174:3300/0 conn(0x7f30c40b5510 0x7f30c40d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c2ffd640 1 -- 172.21.15.62:0/503708329 >> v2:172.21.15.174:3300/0 conn(0x7f30c40b5510 msgr2=0x7f30c40d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c2ffd640 1 --2- 172.21.15.62:0/503708329 >> v2:172.21.15.174:3300/0 conn(0x7f30c40b5510 0x7f30c40d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c2ffd640 1 -- 172.21.15.62:0/503708329 >> v1:172.21.15.62:6789/0 conn(0x7f30c40b58f0 legacy=0x7f30c40d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c2ffd640 1 -- 172.21.15.62:0/503708329 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c40d5c80 con 0x7f30c40b6820 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c27fc640 1 -- 172.21.15.62:0/503708329 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 220773873 0 0) 0x7f30c40c9cc0 con 0x7f30c40b58f0 2024-04-26T22:27:22.104 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.103+0000 7f30c2ffd640 1 --2- 172.21.15.62:0/503708329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c40d1dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b5f00 tx=0x7f30b802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251d17ae9d87cd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c27fc640 1 -- 172.21.15.62:0/503708329 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8037070 con 0x7f30c40b6820 2024-04-26T22:27:22.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c27fc640 1 -- 172.21.15.62:0/503708329 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30b8032440 con 0x7f30c40b6820 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c27fc640 1 -- 172.21.15.62:0/503708329 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b802ae60 con 0x7f30c40b6820 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 -- 172.21.15.62:0/503708329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 msgr2=0x7f30c40d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 --2- 172.21.15.62:0/503708329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c40d1dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b5f00 tx=0x7f30b802a110 comp rx=0 tx=0).stop 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 -- 172.21.15.62:0/503708329 shutdown_connections 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 --2- 172.21.15.62:0/503708329 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c40d1dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 --2- 172.21.15.62:0/503708329 >> v2:172.21.15.174:3300/0 conn(0x7f30c40b5510 0x7f30c40d1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 -- 172.21.15.62:0/503708329 >> 172.21.15.62:0/503708329 conn(0x7f30c40d5160 msgr2=0x7f30c40d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 -- 172.21.15.62:0/503708329 shutdown_connections 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.104+0000 7f30c9be9640 1 -- 172.21.15.62:0/503708329 wait complete. 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.105+0000 7f30c9be9640 1 Processor -- start 2024-04-26T22:27:22.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 -- start start 2024-04-26T22:27:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30c40b5510 0x7f30c40ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.108 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30c40b58f0 0x7f30c40b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40d0b60 con 0x7f30c40b6820 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30c40d0880 con 0x7f30c40b5510 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c9be9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f30c40d4100 con 0x7f30c40b58f0 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35736/0 (socket says 172.21.15.62:35736) 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c3fff640 1 -- 172.21.15.62:0/866504526 learned_addr learned my addr 172.21.15.62:0/866504526 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.106+0000 7f30c2ffd640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30c40b58f0 0x7f30c40b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c37fe640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30c40b5510 0x7f30c40ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 -- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30c40b58f0 msgr2=0x7f30c40b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30c40b58f0 0x7f30c40b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 -- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30c40b5510 msgr2=0x7f30c40ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30c40b5510 0x7f30c40ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 -- 172.21.15.62:0/866504526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c40b8820 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c3fff640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c7a0 tx=0x7f30bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc018070 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30bc00a640 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc00aab0 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c40b8b00 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.107+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c40b97e0 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.108+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c40693e0 con 0x7f30c40b6820 2024-04-26T22:27:22.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.108+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30bc01b070 con 0x7f30c40b6820 2024-04-26T22:27:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.109+0000 7f30abfff640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 0x7f30b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.109+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30bc08b0f0 con 0x7f30c40b6820 2024-04-26T22:27:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.109+0000 7f30c37fe640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 0x7f30b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.110+0000 7f30c37fe640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 0x7f30b006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30b4006fd0 tx=0x7f30b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.113+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30bc0595b0 con 0x7f30c40b6820 2024-04-26T22:27:22.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.297+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f30c40cf690 con 0x7f30c40b6820 2024-04-26T22:27:22.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.298+0000 7f30abfff640 1 -- 172.21.15.62:0/866504526 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f30bc058eb0 con 0x7f30c40b6820 2024-04-26T22:27:22.298 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:22.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.299+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 msgr2=0x7f30b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.299+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 0x7f30b006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30b4006fd0 tx=0x7f30b4008040 comp rx=0 tx=0).stop 2024-04-26T22:27:22.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.299+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 msgr2=0x7f30c415e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.299+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c7a0 tx=0x7f30bc00cc60 comp rx=0 tx=0).stop 2024-04-26T22:27:22.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 shutdown_connections 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30b006c8f0 0x7f30b006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30c40b6820 0x7f30c415e8e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30c40b58f0 0x7f30c40b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 --2- 172.21.15.62:0/866504526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30c40b5510 0x7f30c40ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 >> 172.21.15.62:0/866504526 conn(0x7f30c40d5160 msgr2=0x7f30c40d5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 shutdown_connections 2024-04-26T22:27:22.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.300+0000 7f30c9be9640 1 -- 172.21.15.62:0/866504526 wait complete. 2024-04-26T22:27:22.311 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:22.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 Processor -- start 2024-04-26T22:27:22.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 -- start start 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8afc06fd20 0x7f8afc0b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc0cfa80 con 0x7f8afc0cf530 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc068420 con 0x7f8afc06fd20 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.415+0000 7f8b01c1d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc0cfd60 con 0x7f8afc06f230 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8afc06fd20 0x7f8afc0b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8afc06fd20 0x7f8afc0b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35748/0 (socket says 172.21.15.62:35748) 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 -- 172.21.15.62:0/4134294304 learned_addr learned my addr 172.21.15.62:0/4134294304 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:22.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afb7fe640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 -- 172.21.15.62:0/4134294304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 msgr2=0x7f8afc0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 -- 172.21.15.62:0/4134294304 >> v1:172.21.15.174:6789/0 conn(0x7f8afc0cf530 legacy=0x7f8afc06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.416+0000 7f8afbfff640 1 -- 172.21.15.62:0/4134294304 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc0d0040 con 0x7f8afc06fd20 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8afbfff640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc0b4480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8af400bb10 tx=0x7f8af402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaced1c7d55918f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8afa7fc640 1 -- 172.21.15.62:0/4134294304 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4032790 con 0x7f8afc06fd20 2024-04-26T22:27:22.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8afa7fc640 1 -- 172.21.15.62:0/4134294304 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4038070 con 0x7f8afc06fd20 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8afa7fc640 1 -- 172.21.15.62:0/4134294304 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4032b50 con 0x7f8afc06fd20 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 -- 172.21.15.62:0/4134294304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 msgr2=0x7f8afc0b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc0b4480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8af400bb10 tx=0x7f8af402c8d0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 -- 172.21.15.62:0/4134294304 shutdown_connections 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc0b4480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/4134294304 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 -- 172.21.15.62:0/4134294304 >> 172.21.15.62:0/4134294304 conn(0x7f8afc0ce0b0 msgr2=0x7f8afc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 -- 172.21.15.62:0/4134294304 shutdown_connections 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.417+0000 7f8b01c1d640 1 -- 172.21.15.62:0/4134294304 wait complete. 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 Processor -- start 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 -- start start 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc1f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 0x7f8afc0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.419+0000 7f8b01c1d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0701b0 con 0x7f8afc06fd20 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8b01c1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0d0600 con 0x7f8afc0cf530 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8b01c1d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc15c950 con 0x7f8afc06f230 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:35756/0 (socket says 172.21.15.62:35756) 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 -- 172.21.15.62:0/764922268 learned_addr learned my addr 172.21.15.62:0/764922268 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:22.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afb7fe640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc1f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afbfff640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 0x7f8afc0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 -- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 msgr2=0x7f8afc1f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc1f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 -- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 msgr2=0x7f8afc0c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 0x7f8afc0c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 -- 172.21.15.62:0/764922268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc1f1c70 con 0x7f8afc06fd20 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afb7fe640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc1f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afbfff640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 0x7f8afc0c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8afaffd640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b420 tx=0x7f8af000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011830 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af0016070 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc0c52e0 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011c10 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.420+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc0c7e40 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.421+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8afc0693e0 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.422+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8af001b460 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.422+0000 7f8ae3fff640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.422+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8af008b610 con 0x7f8afc06fd20 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.422+0000 7f8afb7fe640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.423+0000 7f8afb7fe640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 0x7f8ae806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8afc15c2d0 tx=0x7f8aec00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.426+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8af0020020 con 0x7f8afc06fd20 2024-04-26T22:27:22.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.611+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8afc06ea70 con 0x7f8afc06fd20 2024-04-26T22:27:22.612 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.611+0000 7f8ae3fff640 1 -- 172.21.15.62:0/764922268 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8af0059a40 con 0x7f8afc06fd20 2024-04-26T22:27:22.612 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 msgr2=0x7f8ae806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 0x7f8ae806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8afc15c2d0 tx=0x7f8aec00b040 comp rx=0 tx=0).stop 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 msgr2=0x7f8afc15ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8af000b420 tx=0x7f8af000b8e0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 shutdown_connections 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8ae806c8f0 0x7f8ae806edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8afc0cf530 0x7f8afc0c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.614 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8afc06fd20 0x7f8afc15ba40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 --2- 172.21.15.62:0/764922268 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8afc06f230 0x7f8afc1f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.613+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 >> 172.21.15.62:0/764922268 conn(0x7f8afc0ce0b0 msgr2=0x7f8afc15ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.614+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 shutdown_connections 2024-04-26T22:27:22.615 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.614+0000 7f8b01c1d640 1 -- 172.21.15.62:0/764922268 wait complete. 2024-04-26T22:27:22.624 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.724+0000 7f3a3cf46640 1 Processor -- start 2024-04-26T22:27:22.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.724+0000 7f3a3cf46640 1 -- start start 2024-04-26T22:27:22.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.725+0000 7f3a3cf46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a3815f380 0x7f3a380d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.725+0000 7f3a3cf46640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a380d3de0 con 0x7f3a3815f760 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.725+0000 7f3a3cf46640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a380d40c0 con 0x7f3a380b9a90 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.725+0000 7f3a3cf46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a380d43a0 con 0x7f3a3815f380 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a35d9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3a3815f760 0x7f3a380cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46220/0 (socket says 172.21.15.62:46220) 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a35d9b640 1 -- 172.21.15.62:0/3491743893 learned_addr learned my addr 172.21.15.62:0/3491743893 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3659c640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a3815f380 0x7f3a380d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 134281620 0 0) 0x7f3a380d3de0 con 0x7f3a3815f760 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a24003520 con 0x7f3a3815f760 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111467277 0 0) 0x7f3a380d40c0 con 0x7f3a380b9a90 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a380d3de0 con 0x7f3a380b9a90 2024-04-26T22:27:22.729 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2748297909 0 0) 0x7f3a24003520 con 0x7f3a3815f760 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a3815f380 msgr2=0x7f3a380d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a3815f380 0x7f3a380d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 >> v1:172.21.15.174:6789/0 conn(0x7f3a380b9a90 legacy=0x7f3a380d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a380b8280 con 0x7f3a3815f760 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f3a20001670 con 0x7f3a3815f760 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 >> v1:172.21.15.62:6789/0 conn(0x7f3a3815f760 legacy=0x7f3a380cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24003b70 0x7f3a24003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.730 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a240051c0 0x7f3a2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3a380d43a0 con 0x7f3a24004800 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2400bf30 con 0x7f3a24003b70 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2400c130 con 0x7f3a240051c0 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3659c640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a3815f380 0x7f3a380d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a36d9d640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a240051c0 0x7f3a2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.726+0000 7f3a3659c640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24003b70 0x7f3a24003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 -- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a240051c0 msgr2=0x7f3a2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a240051c0 0x7f3a2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 -- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24003b70 msgr2=0x7f3a24003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a24003b70 0x7f3a24003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 -- 172.21.15.62:0/3491743893 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a24003520 con 0x7f3a24004800 2024-04-26T22:27:22.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a24004c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001eb0 tx=0x7f3a24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b580057afa8e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20007ce0 con 0x7f3a24004800 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a20005ad0 con 0x7f3a24004800 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.727+0000 7f3a3559a640 1 -- 172.21.15.62:0/3491743893 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20007070 con 0x7f3a24004800 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 -- 172.21.15.62:0/3491743893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 msgr2=0x7f3a24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/3491743893 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a24004c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a20001eb0 tx=0x7f3a24005830 comp rx=0 tx=0).stop 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a36d9d640 1 -- 172.21.15.62:0/3491743893 reap_dead start 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 -- 172.21.15.62:0/3491743893 shutdown_connections 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 -- 172.21.15.62:0/3491743893 >> 172.21.15.62:0/3491743893 conn(0x7f3a3809e560 msgr2=0x7f3a380d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 -- 172.21.15.62:0/3491743893 shutdown_connections 2024-04-26T22:27:22.732 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.728+0000 7f3a3cf46640 1 -- 172.21.15.62:0/3491743893 wait complete. 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.729+0000 7f3a3cf46640 1 Processor -- start 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.729+0000 7f3a3cf46640 1 -- start start 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a380cd3b0 0x7f3a380cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 0x7f3a380cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3a380d1100 con 0x7f3a24004800 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a380d0cc0 con 0x7f3a380cd7f0 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3cf46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a380d1280 con 0x7f3a380cd3b0 2024-04-26T22:27:22.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51144/0 (socket says 172.21.15.62:51144) 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a3659c640 1 -- 172.21.15.62:0/55810227 learned_addr learned my addr 172.21.15.62:0/55810227 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a36d9d640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 0x7f3a380cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.730+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a380cd3b0 0x7f3a380cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 -- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a380cd3b0 msgr2=0x7f3a380cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a380cd3b0 0x7f3a380cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 -- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 msgr2=0x7f3a380cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 0x7f3a380cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 -- 172.21.15.62:0/55810227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a380cdd50 con 0x7f3a24004800 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a36d9d640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 0x7f3a380cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3659c640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a280119a0 tx=0x7f3a28011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800fbb0 con 0x7f3a24004800 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2800fd50 con 0x7f3a24004800 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a380ce030 con 0x7f3a24004800 2024-04-26T22:27:22.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a28014660 con 0x7f3a24004800 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.731+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a380ce5f0 con 0x7f3a24004800 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.732+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3806a360 con 0x7f3a24004800 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.732+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a28012040 con 0x7f3a24004800 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.733+0000 7f3a2effd640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 0x7f3a1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.733+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a2808c220 con 0x7f3a24004800 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.733+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 0x7f3a1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.733+0000 7f3a35d9b640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 0x7f3a1806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a20032000 tx=0x7f3a20009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:22.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.737+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3a28019030 con 0x7f3a24004800 2024-04-26T22:27:22.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.963+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a380008d0 con 0x7f3a24004800 2024-04-26T22:27:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.963+0000 7f3a2effd640 1 -- 172.21.15.62:0/55810227 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f3a2805a650 con 0x7f3a24004800 2024-04-26T22:27:22.964 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:22.964 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:22.964 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 msgr2=0x7f3a1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 0x7f3a1806ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3a20032000 tx=0x7f3a20009030 comp rx=0 tx=0).stop 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 msgr2=0x7f3a381f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a280119a0 tx=0x7f3a28011e60 comp rx=0 tx=0).stop 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 shutdown_connections 2024-04-26T22:27:22.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3a1806c920 0x7f3a1806ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a380cd7f0 0x7f3a380cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a380cd3b0 0x7f3a380cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 --2- 172.21.15.62:0/55810227 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3a24004800 0x7f3a381f2370 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 >> 172.21.15.62:0/55810227 conn(0x7f3a3809e560 msgr2=0x7f3a3806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 shutdown_connections 2024-04-26T22:27:22.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:22.966+0000 7f3a3cf46640 1 -- 172.21.15.62:0/55810227 wait complete. 2024-04-26T22:27:22.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 Processor -- start 2024-04-26T22:27:23.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 -- start start 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea815f400 0x7f6ea80d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 0x7f6ea80d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80d26c0 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea80d3220 con 0x7f6ea80d0b80 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.078+0000 7f6eae659640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80d5ef0 con 0x7f6ea815f400 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51160/0 (socket says 172.21.15.62:51160) 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 -- 172.21.15.62:0/3315583114 learned_addr learned my addr 172.21.15.62:0/3315583114 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6ea7fff640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea815f400 0x7f6ea80d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3315583114 >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 0x7f6ea80d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 -- 172.21.15.62:0/3315583114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea815f400 msgr2=0x7f6ea80d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea815f400 0x7f6ea80d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 -- 172.21.15.62:0/3315583114 >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 msgr2=0x7f6ea80d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 --2- 172.21.15.62:0/3315583114 >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 0x7f6ea80d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6eacbf6640 1 -- 172.21.15.62:0/3315583114 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80d00a0 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.079+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3315583114 >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 0x7f6ea80d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eacbf6640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ea808e260 tx=0x7f6ea002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a004c53ab327cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6ea6ffd640 1 -- 172.21.15.62:0/3315583114 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0036070 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6ea6ffd640 1 -- 172.21.15.62:0/3315583114 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0031600 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6ea6ffd640 1 -- 172.21.15.62:0/3315583114 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00319d0 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 -- 172.21.15.62:0/3315583114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0fc0 msgr2=0x7f6ea80d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ea808e260 tx=0x7f6ea002a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 -- 172.21.15.62:0/3315583114 shutdown_connections 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0fc0 0x7f6ea80d3c80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 --2- 172.21.15.62:0/3315583114 >> v2:172.21.15.174:3300/0 conn(0x7f6ea80d0b80 0x7f6ea80d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 --2- 172.21.15.62:0/3315583114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea815f400 0x7f6ea80d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 -- 172.21.15.62:0/3315583114 >> 172.21.15.62:0/3315583114 conn(0x7f6ea809e510 msgr2=0x7f6ea80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 -- 172.21.15.62:0/3315583114 shutdown_connections 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.080+0000 7f6eae659640 1 -- 172.21.15.62:0/3315583114 wait complete. 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 Processor -- start 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 -- start start 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 0x7f6ea80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 0x7f6ea8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80cfd50 con 0x7f6ea80d0b80 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea80d0660 con 0x7f6ea815f400 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.082+0000 7f6eae659640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea815f7e0 con 0x7f6ea80d0fc0 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51176/0 (socket says 172.21.15.62:51176) 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 -- 172.21.15.62:0/3831758840 learned_addr learned my addr 172.21.15.62:0/3831758840 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6eacbf6640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 0x7f6ea8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 0x7f6ea80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 -- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 msgr2=0x7f6ea80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 0x7f6ea80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 -- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 msgr2=0x7f6ea8151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 0x7f6ea8151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 -- 172.21.15.62:0/3831758840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea80ba850 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 0x7f6ea80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6eacbf6640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 0x7f6ea8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea7fff640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800d830 tx=0x7f6e9800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800b6c0 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e98019070 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea80bab30 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800ba80 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.083+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea8160790 con 0x7f6ea80d0b80 2024-04-26T22:27:23.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.084+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea80bad00 con 0x7f6ea80d0b80 2024-04-26T22:27:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.085+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e9801e460 con 0x7f6ea80d0b80 2024-04-26T22:27:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.085+0000 7f6ea4ff9640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.085+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e9808c220 con 0x7f6ea80d0b80 2024-04-26T22:27:23.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.085+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.086+0000 7f6ea77fe640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 0x7f6e9406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006fd0 tx=0x7f6e9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.089+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6e9805a650 con 0x7f6ea80d0b80 2024-04-26T22:27:23.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.275+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ea80d2880 con 0x7f6ea80d0b80 2024-04-26T22:27:23.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.276+0000 7f6ea4ff9640 1 -- 172.21.15.62:0/3831758840 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f6e98059f50 con 0x7f6ea80d0b80 2024-04-26T22:27:23.277 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:23.277 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 msgr2=0x7f6e9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 0x7f6e9406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006fd0 tx=0x7f6e9c008040 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 msgr2=0x7f6ea80b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800d830 tx=0x7f6e9800dcf0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 shutdown_connections 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6e9406c8f0 0x7f6e9406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ea815f400 0x7f6ea8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6ea80d0fc0 0x7f6ea80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 --2- 172.21.15.62:0/3831758840 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ea80d0b80 0x7f6ea80b9f00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.278+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 >> 172.21.15.62:0/3831758840 conn(0x7f6ea809e510 msgr2=0x7f6ea80b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.279+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 shutdown_connections 2024-04-26T22:27:23.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.279+0000 7f6eae659640 1 -- 172.21.15.62:0/3831758840 wait complete. 2024-04-26T22:27:23.290 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:23.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:23.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.393+0000 7fc7ca459640 1 Processor -- start 2024-04-26T22:27:23.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.393+0000 7fc7ca459640 1 -- start start 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7ca459640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc7c40d5a80 0x7fc7c40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7ca459640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c40b2760 con 0x7fc7c40b4b20 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7ca459640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c40b2a00 con 0x7fc7c415f400 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7ca459640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c40b43e0 con 0x7fc7c40d5a80 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c89f6640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc7c40b4b20 0x7fc7c41630b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46236/0 (socket says 172.21.15.62:46236) 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c89f6640 1 -- 172.21.15.62:0/2815465583 learned_addr learned my addr 172.21.15.62:0/2815465583 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c37fe640 1 --2- 172.21.15.62:0/2815465583 >> v2:172.21.15.174:3300/0 conn(0x7fc7c40d5a80 0x7fc7c40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762361132 0 0) 0x7fc7c40b2760 con 0x7fc7c40b4b20 2024-04-26T22:27:23.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7ac003520 con 0x7fc7c40b4b20 2024-04-26T22:27:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c37fe640 1 -- 172.21.15.62:0/2815465583 >> v1:172.21.15.174:6789/0 conn(0x7fc7c415f400 legacy=0x7fc7c40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2889628156 0 0) 0x7fc7ac003520 con 0x7fc7c40b4b20 2024-04-26T22:27:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c37fe640 1 -- 172.21.15.62:0/2815465583 >> v1:172.21.15.62:6789/0 conn(0x7fc7c40b4b20 legacy=0x7fc7c41630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.394+0000 7fc7c37fe640 1 -- 172.21.15.62:0/2815465583 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c40b2ce0 con 0x7fc7c40d5a80 2024-04-26T22:27:23.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.395+0000 7fc7c37fe640 1 --2- 172.21.15.62:0/2815465583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c40d5a80 0x7fc7c40b45e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009720 tx=0x7fc7b4029fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849dc1da0fceea97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.395+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4037070 con 0x7fc7c40d5a80 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.395+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b402aa60 con 0x7fc7c40d5a80 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.395+0000 7fc7c2ffd640 1 -- 172.21.15.62:0/2815465583 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc001a10 con 0x7fc7c40d5a80 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 -- 172.21.15.62:0/2815465583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c40d5a80 msgr2=0x7fc7c40b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 --2- 172.21.15.62:0/2815465583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c40d5a80 0x7fc7c40b45e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009720 tx=0x7fc7b4029fd0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 -- 172.21.15.62:0/2815465583 shutdown_connections 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 --2- 172.21.15.62:0/2815465583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c40d5a80 0x7fc7c40b45e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 -- 172.21.15.62:0/2815465583 >> 172.21.15.62:0/2815465583 conn(0x7fc7c409e4f0 msgr2=0x7fc7c40d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 -- 172.21.15.62:0/2815465583 shutdown_connections 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.396+0000 7fc7ca459640 1 -- 172.21.15.62:0/2815465583 wait complete. 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.397+0000 7fc7ca459640 1 Processor -- start 2024-04-26T22:27:23.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.397+0000 7fc7ca459640 1 -- start start 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7c40d5a80 0x7fc7c41609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c415f400 0x7fc7c40c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40b32a0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40b3ed0 con 0x7fc7c415f400 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7ca459640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c40b4f30 con 0x7fc7c40d5a80 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51186/0 (socket says 172.21.15.62:51186) 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 -- 172.21.15.62:0/4041686443 learned_addr learned my addr 172.21.15.62:0/4041686443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c37fe640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7c40d5a80 0x7fc7c41609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c89f6640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c415f400 0x7fc7c40c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 -- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7c40d5a80 msgr2=0x7fc7c41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7c40d5a80 0x7fc7c41609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 -- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c415f400 msgr2=0x7fc7c40c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c415f400 0x7fc7c40c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.398+0000 7fc7c3fff640 1 -- 172.21.15.62:0/4041686443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c4161430 con 0x7fc7c40b4b20 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7c3fff640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c750 tx=0x7fc7b800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8017070 con 0x7fc7c40b4b20 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b8005270 con 0x7fc7c40b4b20 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c4161710 con 0x7fc7c40b4b20 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800a640 con 0x7fc7c40b4b20 2024-04-26T22:27:23.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.399+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c4161cd0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.400+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c406a360 con 0x7fc7c40b4b20 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.400+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7b801a070 con 0x7fc7c40b4b20 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.400+0000 7fc7c0ff9640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 0x7fc7b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.401+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7b808b0a0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.401+0000 7fc7c37fe640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 0x7fc7b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.401+0000 7fc7c37fe640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 0x7fc7b006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4034000 tx=0x7fc7b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.405+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7b80594d0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.589+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc7c40d28f0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.590+0000 7fc7c0ff9640 1 -- 172.21.15.62:0/4041686443 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc7b8058dd0 con 0x7fc7c40b4b20 2024-04-26T22:27:23.595 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:23.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 msgr2=0x7fc7b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 0x7fc7b006edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4034000 tx=0x7fc7b402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 msgr2=0x7fc7c4159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c750 tx=0x7fc7b800cc10 comp rx=0 tx=0).stop 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 shutdown_connections 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc7b006c8f0 0x7fc7b006edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7c415f400 0x7fc7c40c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc7c40d5a80 0x7fc7c41609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 --2- 172.21.15.62:0/4041686443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc7c40b4b20 0x7fc7c4159380 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 >> 172.21.15.62:0/4041686443 conn(0x7fc7c409e4f0 msgr2=0x7fc7c40d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 shutdown_connections 2024-04-26T22:27:23.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.592+0000 7fc7ca459640 1 -- 172.21.15.62:0/4041686443 wait complete. 2024-04-26T22:27:23.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:23.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.715+0000 7f199a778640 1 Processor -- start 2024-04-26T22:27:23.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.715+0000 7f199a778640 1 -- start start 2024-04-26T22:27:23.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f199a778640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f199415cdf0 0x7f19940d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f199a778640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f199a778640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19940d2590 con 0x7f19940d32d0 2024-04-26T22:27:23.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f199a778640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f19940d4310 con 0x7f199415cdf0 2024-04-26T22:27:23.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f199a778640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f19940d2870 con 0x7f199415d1d0 2024-04-26T22:27:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1993fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f199415cdf0 0x7f19940d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56892/0 (socket says 172.21.15.62:56892) 2024-04-26T22:27:23.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 -- 172.21.15.62:0/437440681 learned_addr learned my addr 172.21.15.62:0/437440681 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f19937fe640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f19940d32d0 0x7f19940d1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:48500/0 (socket says 172.21.15.62:48500) 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 -- 172.21.15.62:0/437440681 >> v2:172.21.15.174:3300/0 conn(0x7f199415cdf0 msgr2=0x7f19940d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 --2- 172.21.15.62:0/437440681 >> v2:172.21.15.174:3300/0 conn(0x7f199415cdf0 0x7f19940d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.716+0000 7f1998d15640 1 -- 172.21.15.62:0/437440681 >> v1:172.21.15.174:6789/0 conn(0x7f19940d32d0 legacy=0x7f19940d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.717+0000 7f1998d15640 1 -- 172.21.15.62:0/437440681 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19940b50b0 con 0x7f199415d1d0 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.717+0000 7f1998d15640 1 --2- 172.21.15.62:0/437440681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f198c009e50 tx=0x7f198c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1eb366d4af4b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.717+0000 7f1992ffd640 1 -- 172.21.15.62:0/437440681 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c036070 con 0x7f199415d1d0 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.717+0000 7f1992ffd640 1 -- 172.21.15.62:0/437440681 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c031600 con 0x7f199415d1d0 2024-04-26T22:27:23.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.717+0000 7f1992ffd640 1 -- 172.21.15.62:0/437440681 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0319d0 con 0x7f199415d1d0 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 -- 172.21.15.62:0/437440681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 msgr2=0x7f19940b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 --2- 172.21.15.62:0/437440681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f198c009e50 tx=0x7f198c02a710 comp rx=0 tx=0).stop 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 -- 172.21.15.62:0/437440681 shutdown_connections 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 --2- 172.21.15.62:0/437440681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415d1d0 0x7f19940b58b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 --2- 172.21.15.62:0/437440681 >> v2:172.21.15.174:3300/0 conn(0x7f199415cdf0 0x7f19940d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 -- 172.21.15.62:0/437440681 >> 172.21.15.62:0/437440681 conn(0x7f199408ec30 msgr2=0x7f199409e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 -- 172.21.15.62:0/437440681 shutdown_connections 2024-04-26T22:27:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.718+0000 7f199a778640 1 -- 172.21.15.62:0/437440681 wait complete. 2024-04-26T22:27:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.719+0000 7f199a778640 1 Processor -- start 2024-04-26T22:27:23.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.719+0000 7f199a778640 1 -- start start 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19940d32d0 0x7f19940ba650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415cdf0 0x7f19940c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19940d21e0 con 0x7f199415d1d0 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f199415fcc0 con 0x7f19940d32d0 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f199a778640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f19940cb480 con 0x7f199415cdf0 2024-04-26T22:27:23.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f1998d15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f1998d15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51190/0 (socket says 172.21.15.62:51190) 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f1998d15640 1 -- 172.21.15.62:0/2045723911 learned_addr learned my addr 172.21.15.62:0/2045723911 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f19937fe640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415cdf0 0x7f19940c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.720+0000 7f1993fff640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19940d32d0 0x7f19940ba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 -- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415cdf0 msgr2=0x7f19940c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415cdf0 0x7f19940c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 -- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19940d32d0 msgr2=0x7f19940ba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19940d32d0 0x7f19940ba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 -- 172.21.15.62:0/2045723911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19941f5d40 con 0x7f199415d1d0 2024-04-26T22:27:23.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1998d15640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f198c009f70 tx=0x7f198c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c03f020 con 0x7f199415d1d0 2024-04-26T22:27:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19940bab90 con 0x7f199415d1d0 2024-04-26T22:27:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f198c037040 con 0x7f199415d1d0 2024-04-26T22:27:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c036070 con 0x7f199415d1d0 2024-04-26T22:27:23.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.721+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19940cb9d0 con 0x7f199415d1d0 2024-04-26T22:27:23.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.722+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199406a360 con 0x7f199415d1d0 2024-04-26T22:27:23.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.722+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f198c042020 con 0x7f199415d1d0 2024-04-26T22:27:23.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.723+0000 7f1990ff9640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 0x7f198006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:23.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.723+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f198c0abe30 con 0x7f199415d1d0 2024-04-26T22:27:23.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.723+0000 7f1993fff640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 0x7f198006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:23.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.723+0000 7f1993fff640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 0x7f198006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19940c8750 tx=0x7f1988009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:23.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.727+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f198c049030 con 0x7f199415d1d0 2024-04-26T22:27:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.912+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19940b4cd0 con 0x7f199415d1d0 2024-04-26T22:27:23.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.913+0000 7f1990ff9640 1 -- 172.21.15.62:0/2045723911 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f198c079c20 con 0x7f199415d1d0 2024-04-26T22:27:23.913 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 msgr2=0x7f198006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 0x7f198006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19940c8750 tx=0x7f1988009190 comp rx=0 tx=0).stop 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 msgr2=0x7f19941616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f198c009f70 tx=0x7f198c004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 shutdown_connections 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f198006c8f0 0x7f198006edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f199415d1d0 0x7f19941616b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f199415cdf0 0x7f19940c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 --2- 172.21.15.62:0/2045723911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19940d32d0 0x7f19940ba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:23.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 >> 172.21.15.62:0/2045723911 conn(0x7f199408ec30 msgr2=0x7f19940cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:23.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 shutdown_connections 2024-04-26T22:27:23.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:23.915+0000 7f199a778640 1 -- 172.21.15.62:0/2045723911 wait complete. 2024-04-26T22:27:23.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.029+0000 7fe34acc5640 1 Processor -- start 2024-04-26T22:27:24.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.029+0000 7fe34acc5640 1 -- start start 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe34acc5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe3440d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe34acc5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe3440d0d30 0x7fe3440d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe34acc5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3440d2ed0 con 0x7fe3440d1fe0 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe34acc5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe3440d1190 con 0x7fe3440d0d30 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe34acc5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe3440d31b0 con 0x7fe34415f470 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe348a61640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe3440d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe3440d0d30 0x7fe3440d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe3440d0d30 0x7fe3440d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:35240/0 (socket says 172.21.15.62:35240) 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe348a61640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe3440d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56916/0 (socket says 172.21.15.62:56916) 2024-04-26T22:27:24.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 -- 172.21.15.62:0/2361918666 learned_addr learned my addr 172.21.15.62:0/2361918666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe349262640 1 --1- 172.21.15.62:0/2361918666 >> v1:172.21.15.174:6789/0 conn(0x7fe3440d1fe0 0x7fe344162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:48506/0 (socket says 172.21.15.62:48506) 2024-04-26T22:27:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 -- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 msgr2=0x7fe3440d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 --2- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe3440d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 -- 172.21.15.62:0/2361918666 >> v1:172.21.15.174:6789/0 conn(0x7fe3440d1fe0 legacy=0x7fe344162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.030+0000 7fe33bfff640 1 -- 172.21.15.62:0/2361918666 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3440d3490 con 0x7fe3440d0d30 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.031+0000 7fe33bfff640 1 --2- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe3440d1aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098a0 tx=0x7fe33402a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e89fdf570d86a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.031+0000 7fe33b7fe640 1 -- 172.21.15.62:0/2361918666 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334037070 con 0x7fe3440d0d30 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.031+0000 7fe33b7fe640 1 -- 172.21.15.62:0/2361918666 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe334032440 con 0x7fe3440d0d30 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.031+0000 7fe33b7fe640 1 -- 172.21.15.62:0/2361918666 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334032810 con 0x7fe3440d0d30 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 -- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 msgr2=0x7fe3440d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 --2- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe3440d1aa0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe3340098a0 tx=0x7fe33402a290 comp rx=0 tx=0).stop 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 -- 172.21.15.62:0/2361918666 shutdown_connections 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 --2- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe3440d1aa0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 --2- 172.21.15.62:0/2361918666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe3440d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 -- 172.21.15.62:0/2361918666 >> 172.21.15.62:0/2361918666 conn(0x7fe34409e510 msgr2=0x7fe3440d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 -- 172.21.15.62:0/2361918666 shutdown_connections 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.032+0000 7fe34acc5640 1 -- 172.21.15.62:0/2361918666 wait complete. 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.033+0000 7fe34acc5640 1 Processor -- start 2024-04-26T22:27:24.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.033+0000 7fe34acc5640 1 -- start start 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe344158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe34415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe34415f850 con 0x7fe3440d1fe0 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3440d3a50 con 0x7fe3440d0d30 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe34acc5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe3440d3f70 con 0x7fe34415f470 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51204/0 (socket says 172.21.15.62:51204) 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 -- 172.21.15.62:0/713299198 learned_addr learned my addr 172.21.15.62:0/713299198 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe348a61640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe344158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe349262640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe34415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 -- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 msgr2=0x7fe34415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe34415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 -- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 msgr2=0x7fe344158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe344158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe33bfff640 1 -- 172.21.15.62:0/713299198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3440ce6f0 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.034+0000 7fe348a61640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe344158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe33bfff640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe334000c00 tx=0x7fe3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334041020 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33403d620 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3440ce9d0 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334037040 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.035+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3440cef90 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.036+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34406a360 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.036+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe334044020 con 0x7fe3440d1fe0 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.037+0000 7fe3397fa640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 0x7fe33006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.037+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3340ac230 con 0x7fe3440d1fe0 2024-04-26T22:27:24.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.037+0000 7fe348a61640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 0x7fe33006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.037+0000 7fe348a61640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 0x7fe33006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe340009af0 tx=0x7fe340008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.041+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe33404b030 con 0x7fe3440d1fe0 2024-04-26T22:27:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.225+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe3440d5870 con 0x7fe3440d1fe0 2024-04-26T22:27:24.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.225+0000 7fe3397fa640 1 -- 172.21.15.62:0/713299198 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe33407a670 con 0x7fe3440d1fe0 2024-04-26T22:27:24.225 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 msgr2=0x7fe33006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 0x7fe33006edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe340009af0 tx=0x7fe340008040 comp rx=0 tx=0).stop 2024-04-26T22:27:24.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 msgr2=0x7fe34415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe334000c00 tx=0x7fe3340046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 shutdown_connections 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe33006c8f0 0x7fe33006edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe34415f470 0x7fe34415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3440d1fe0 0x7fe34415b970 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 --2- 172.21.15.62:0/713299198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3440d0d30 0x7fe344158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 >> 172.21.15.62:0/713299198 conn(0x7fe34409e510 msgr2=0x7fe3440d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 shutdown_connections 2024-04-26T22:27:24.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.227+0000 7fe34acc5640 1 -- 172.21.15.62:0/713299198 wait complete. 2024-04-26T22:27:24.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:24.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.339+0000 7fd83e7af640 1 Processor -- start 2024-04-26T22:27:24.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.339+0000 7fd83e7af640 1 -- start start 2024-04-26T22:27:24.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83e7af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd8380d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83e7af640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8380b9a90 0x7fd8380d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83e7af640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8380d3de0 con 0x7fd83815f760 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83e7af640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd8380d43a0 con 0x7fd8380b9a90 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83e7af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8380d40c0 con 0x7fd83815f380 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8380b9a90 0x7fd8380d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd8380b9a90 0x7fd8380d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51216/0 (socket says 172.21.15.62:51216) 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 -- 172.21.15.62:0/860936255 learned_addr learned my addr 172.21.15.62:0/860936255 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd837fff640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd8380d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 -- 172.21.15.62:0/860936255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 msgr2=0x7fd8380d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:24.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd8380d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 -- 172.21.15.62:0/860936255 >> v1:172.21.15.174:6789/0 conn(0x7fd83815f760 legacy=0x7fd8380cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.340+0000 7fd83cd4c640 1 -- 172.21.15.62:0/860936255 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8380b8280 con 0x7fd8380b9a90 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.341+0000 7fd83cd4c640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8380b9a90 0x7fd8380d2110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd830009e50 tx=0x7fd83002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a9aa357d265901 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.341+0000 7fd836ffd640 1 -- 172.21.15.62:0/860936255 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830036070 con 0x7fd8380b9a90 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.341+0000 7fd836ffd640 1 -- 172.21.15.62:0/860936255 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd830031600 con 0x7fd8380b9a90 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.341+0000 7fd836ffd640 1 -- 172.21.15.62:0/860936255 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8300319d0 con 0x7fd8380b9a90 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 -- 172.21.15.62:0/860936255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8380b9a90 msgr2=0x7fd8380d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8380b9a90 0x7fd8380d2110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd830009e50 tx=0x7fd83002a7a0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 -- 172.21.15.62:0/860936255 shutdown_connections 2024-04-26T22:27:24.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8380b9a90 0x7fd8380d2110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 --2- 172.21.15.62:0/860936255 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd8380d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 -- 172.21.15.62:0/860936255 >> 172.21.15.62:0/860936255 conn(0x7fd83809e560 msgr2=0x7fd8380d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 -- 172.21.15.62:0/860936255 shutdown_connections 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.342+0000 7fd83e7af640 1 -- 172.21.15.62:0/860936255 wait complete. 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.343+0000 7fd83e7af640 1 Processor -- start 2024-04-26T22:27:24.344 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.343+0000 7fd83e7af640 1 -- start start 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8380b9a90 0x7fd8380cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd838151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8380d1280 con 0x7fd83815f760 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8380d1100 con 0x7fd8380b9a90 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83e7af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8380d5b50 con 0x7fd83815f380 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51232/0 (socket says 172.21.15.62:51232) 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 -- 172.21.15.62:0/3801445020 learned_addr learned my addr 172.21.15.62:0/3801445020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:24.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd837fff640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8380b9a90 0x7fd8380cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd8377fe640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd838151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 -- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 msgr2=0x7fd838151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd838151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 -- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8380b9a90 msgr2=0x7fd8380cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8380b9a90 0x7fd8380cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.344+0000 7fd83cd4c640 1 -- 172.21.15.62:0/3801445020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8380cf1a0 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd83cd4c640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd830034040 tx=0x7fd830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830036050 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd830042070 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8380cd710 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830009510 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.345+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8380cd9b0 con 0x7fd83815f760 2024-04-26T22:27:24.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.346+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83806a360 con 0x7fd83815f760 2024-04-26T22:27:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.346+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd830007bf0 con 0x7fd83815f760 2024-04-26T22:27:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.347+0000 7fd834ff9640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.347+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8300ac800 con 0x7fd83815f760 2024-04-26T22:27:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.347+0000 7fd837fff640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.348+0000 7fd837fff640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 0x7fd82406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009b40 tx=0x7fd82c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.351+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd83007a5e0 con 0x7fd83815f760 2024-04-26T22:27:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.574+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8380b6820 con 0x7fd83815f760 2024-04-26T22:27:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.575+0000 7fd834ff9640 1 -- 172.21.15.62:0/3801445020 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fd83007e8e0 con 0x7fd83815f760 2024-04-26T22:27:24.575 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:24.575 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:24.575 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 msgr2=0x7fd82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 0x7fd82406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009b40 tx=0x7fd82c009190 comp rx=0 tx=0).stop 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 msgr2=0x7fd8380cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd830034040 tx=0x7fd830004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 shutdown_connections 2024-04-26T22:27:24.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd82406c8f0 0x7fd82406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd83815f760 0x7fd8380cc360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd83815f380 0x7fd838151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 --2- 172.21.15.62:0/3801445020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8380b9a90 0x7fd8380cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 >> 172.21.15.62:0/3801445020 conn(0x7fd83809e560 msgr2=0x7fd83806ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 shutdown_connections 2024-04-26T22:27:24.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.577+0000 7fd83e7af640 1 -- 172.21.15.62:0/3801445020 wait complete. 2024-04-26T22:27:24.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:24.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.688+0000 7f06ae5f2640 1 Processor -- start 2024-04-26T22:27:24.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.688+0000 7f06ae5f2640 1 -- start start 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06ae5f2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f06a80caff0 0x7f06a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06ae5f2640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a806c3e0 con 0x7f06a815f460 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06ae5f2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a806c6c0 con 0x7f06a815f080 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06ae5f2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f06a806d9c0 con 0x7f06a80caff0 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a77fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f06a815f460 0x7f06a80ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46246/0 (socket says 172.21.15.62:46246) 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a77fe640 1 -- 172.21.15.62:0/2980261017 learned_addr learned my addr 172.21.15.62:0/2980261017 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06acb8f640 1 --2- 172.21.15.62:0/2980261017 >> v2:172.21.15.174:3300/0 conn(0x7f06a80caff0 0x7f06a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2355024767 0 0) 0x7f06a806c3e0 con 0x7f06a815f460 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0690003520 con 0x7f06a815f460 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216390674 0 0) 0x7f0690003520 con 0x7f06a815f460 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 >> v2:172.21.15.174:3300/0 conn(0x7f06a80caff0 msgr2=0x7f06a806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 --2- 172.21.15.62:0/2980261017 >> v2:172.21.15.174:3300/0 conn(0x7f06a80caff0 0x7f06a806d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 >> v1:172.21.15.174:6789/0 conn(0x7f06a815f080 legacy=0x7f06a80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a806c9a0 con 0x7f06a815f460 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.689+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f0698001670 con 0x7f06a815f460 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 >> v1:172.21.15.62:6789/0 conn(0x7f06a815f460 legacy=0x7f06a80ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900039b0 0x7f0690003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0690004800 0x7f0690004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 0x7f069000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06a806d9c0 con 0x7f06900051c0 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f069000c160 con 0x7f0690004800 2024-04-26T22:27:24.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f069000c360 con 0x7f06900039b0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 0x7f069000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a7fff640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900039b0 0x7f0690003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06a77fe640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0690004800 0x7f0690004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 -- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900039b0 msgr2=0x7f0690003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900039b0 0x7f0690003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 -- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0690004800 msgr2=0x7f0690004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0690004800 0x7f0690004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.690+0000 7f06acb8f640 1 -- 172.21.15.62:0/2980261017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0690003520 con 0x7f06900051c0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06acb8f640 1 -- 172.21.15.62:0/2980261017 reap_dead start 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06acb8f640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 0x7f069000b9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009fc0 tx=0x7f0690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b7c52a11074a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035020 con 0x7f06900051c0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a0003bd0 con 0x7f06900051c0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06a6ffd640 1 -- 172.21.15.62:0/2980261017 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0009c10 con 0x7f06900051c0 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 -- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 msgr2=0x7f069000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 0x7f069000b9c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009fc0 tx=0x7f0690005830 comp rx=0 tx=0).stop 2024-04-26T22:27:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 -- 172.21.15.62:0/2980261017 shutdown_connections 2024-04-26T22:27:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/2980261017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06900051c0 0x7f069000b9c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 -- 172.21.15.62:0/2980261017 >> 172.21.15.62:0/2980261017 conn(0x7f06a80ce050 msgr2=0x7f06a80ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 -- 172.21.15.62:0/2980261017 shutdown_connections 2024-04-26T22:27:24.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.691+0000 7f06ae5f2640 1 -- 172.21.15.62:0/2980261017 wait complete. 2024-04-26T22:27:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 Processor -- start 2024-04-26T22:27:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 -- start start 2024-04-26T22:27:24.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900051c0 0x7f06a8151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06a8147360 0x7f06a8144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f06a80b0e00 con 0x7f06a80b6a40 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06a806cf60 con 0x7f06a8147360 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.693+0000 7f06ae5f2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f06a80b0f80 con 0x7f06900051c0 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51240/0 (socket says 172.21.15.62:51240) 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 -- 172.21.15.62:0/3621398620 learned_addr learned my addr 172.21.15.62:0/3621398620 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06a7fff640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900051c0 0x7f06a8151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06a77fe640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06a8147360 0x7f06a8144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 -- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900051c0 msgr2=0x7f06a8151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900051c0 0x7f06a8151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 -- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06a8147360 msgr2=0x7f06a8144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06a8147360 0x7f06a8144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 -- 172.21.15.62:0/3621398620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a806ff00 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06acb8f640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06a000e230 tx=0x7f06a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a0035030 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a0043070 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a80701e0 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a002d070 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.694+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a80707a0 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.695+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a806a360 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.696+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06a0005d10 con 0x7f06a80b6a40 2024-04-26T22:27:24.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.696+0000 7f06a4ff9640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.696+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06a00ac9c0 con 0x7f06a80b6a40 2024-04-26T22:27:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.696+0000 7f06a7fff640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.697+0000 7f06a7fff640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:24.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.700+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06a00aee20 con 0x7f06a80b6a40 2024-04-26T22:27:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.885+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06a8070c40 con 0x7f06a80b6a40 2024-04-26T22:27:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.886+0000 7f06a4ff9640 1 -- 172.21.15.62:0/3621398620 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f06a007adf0 con 0x7f06a80b6a40 2024-04-26T22:27:24.886 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:24.887 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 msgr2=0x7f069406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f069c000c00 tx=0x7f069c009040 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 msgr2=0x7f06a80b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06a000e230 tx=0x7f06a0009750 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 shutdown_connections 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f069406c8f0 0x7f069406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f06a80b6a40 0x7f06a80b6ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06a8147360 0x7f06a8144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 --2- 172.21.15.62:0/3621398620 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06900051c0 0x7f06a8151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 >> 172.21.15.62:0/3621398620 conn(0x7f06a80ce050 msgr2=0x7f06a8162ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 shutdown_connections 2024-04-26T22:27:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:24.888+0000 7f06ae5f2640 1 -- 172.21.15.62:0/3621398620 wait complete. 2024-04-26T22:27:24.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.000+0000 7fa672ac8640 1 Processor -- start 2024-04-26T22:27:25.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.000+0000 7fa672ac8640 1 -- start start 2024-04-26T22:27:25.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa66c15d420 0x7fa66c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c0d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa66c0d1b10 0x7fa66c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa66c0d1f70 con 0x7fa66c0d1b10 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa66c0b28b0 con 0x7fa66c15d420 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa672ac8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c0ceda0 con 0x7fa66c0ce630 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa66c0d1b10 0x7fa66c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa66c0d1b10 0x7fa66c0cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51256/0 (socket says 172.21.15.62:51256) 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 -- 172.21.15.62:0/1997426452 learned_addr learned my addr 172.21.15.62:0/1997426452 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa663fff640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c0d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa670864640 1 --2- 172.21.15.62:0/1997426452 >> v2:172.21.15.174:3300/0 conn(0x7fa66c15d420 0x7fa66c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 -- 172.21.15.62:0/1997426452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 msgr2=0x7fa66c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c0d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 -- 172.21.15.62:0/1997426452 >> v2:172.21.15.174:3300/0 conn(0x7fa66c15d420 msgr2=0x7fa66c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 --2- 172.21.15.62:0/1997426452 >> v2:172.21.15.174:3300/0 conn(0x7fa66c15d420 0x7fa66c0d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.001+0000 7fa671065640 1 -- 172.21.15.62:0/1997426452 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c0b2b40 con 0x7fa66c0d1b10 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa671065640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c0d1b10 0x7fa66c0cf850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa66c073590 tx=0x7fa66402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0d52d2833f091e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa6637fe640 1 -- 172.21.15.62:0/1997426452 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664036070 con 0x7fa66c0d1b10 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa6637fe640 1 -- 172.21.15.62:0/1997426452 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa664031600 con 0x7fa66c0d1b10 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa6637fe640 1 -- 172.21.15.62:0/1997426452 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640319d0 con 0x7fa66c0d1b10 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa672ac8640 1 -- 172.21.15.62:0/1997426452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c0d1b10 msgr2=0x7fa66c0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.002+0000 7fa672ac8640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c0d1b10 0x7fa66c0cf850 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa66c073590 tx=0x7fa66402a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 -- 172.21.15.62:0/1997426452 shutdown_connections 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c0d1b10 0x7fa66c0cf850 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 --2- 172.21.15.62:0/1997426452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c0d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 --2- 172.21.15.62:0/1997426452 >> v2:172.21.15.174:3300/0 conn(0x7fa66c15d420 0x7fa66c0d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 -- 172.21.15.62:0/1997426452 >> 172.21.15.62:0/1997426452 conn(0x7fa66c0d3010 msgr2=0x7fa66c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 -- 172.21.15.62:0/1997426452 shutdown_connections 2024-04-26T22:27:25.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.003+0000 7fa672ac8640 1 -- 172.21.15.62:0/1997426452 wait complete. 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.004+0000 7fa672ac8640 1 Processor -- start 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.004+0000 7fa672ac8640 1 -- start start 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa66c0d1b10 0x7fa66c06dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0cea90 con 0x7fa66c15d420 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0b3100 con 0x7fa66c0d1b10 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c0cfd90 con 0x7fa66c0ce630 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51266/0 (socket says 172.21.15.62:51266) 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 -- 172.21.15.62:0/3248463140 learned_addr learned my addr 172.21.15.62:0/3248463140 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa670864640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa663fff640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa66c0d1b10 0x7fa66c06dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 -- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 msgr2=0x7fa66c1546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c1546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 -- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa66c0d1b10 msgr2=0x7fa66c06dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa66c0d1b10 0x7fa66c06dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 -- 172.21.15.62:0/3248463140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c15d840 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa671065640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa664009f40 tx=0x7fa664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66403f020 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa664037040 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c06e5b0 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664036070 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.005+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c06eb70 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.006+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c0693e0 con 0x7fa66c15d420 2024-04-26T22:27:25.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.007+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa664042020 con 0x7fa66c15d420 2024-04-26T22:27:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.007+0000 7fa6617fa640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 0x7fa65806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.008+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa6640ac230 con 0x7fa66c15d420 2024-04-26T22:27:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.008+0000 7fa670864640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 0x7fa65806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.008+0000 7fa670864640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 0x7fa65806ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa668009a30 tx=0x7fa668008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.012+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa664049020 con 0x7fa66c15d420 2024-04-26T22:27:25.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.236+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa66c071a60 con 0x7fa66c15d420 2024-04-26T22:27:25.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.237+0000 7fa6617fa640 1 -- 172.21.15.62:0/3248463140 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fa66407a5c0 con 0x7fa66c15d420 2024-04-26T22:27:25.237 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:25.237 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:25.237 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 msgr2=0x7fa65806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 0x7fa65806ede0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa668009a30 tx=0x7fa668008040 comp rx=0 tx=0).stop 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 msgr2=0x7fa66c15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa664009f40 tx=0x7fa664004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 shutdown_connections 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa65806c920 0x7fa65806ede0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa66c15d420 0x7fa66c15fcd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa66c0d1b10 0x7fa66c06dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 --2- 172.21.15.62:0/3248463140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa66c0ce630 0x7fa66c1546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 >> 172.21.15.62:0/3248463140 conn(0x7fa66c0d3010 msgr2=0x7fa66c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 shutdown_connections 2024-04-26T22:27:25.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.239+0000 7fa672ac8640 1 -- 172.21.15.62:0/3248463140 wait complete. 2024-04-26T22:27:25.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T22:27:13.221716+0000', 'modified': '2024-04-26T22:27:16.231055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6835/3095900185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6833', 'nonce': 3095900185}, {'type': 'v1', 'addr': '172.21.15.62:6835', 'nonce': 3095900185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4287, 'qdb_cluster': [4287]} 2024-04-26T22:27:25.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:27:25.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T22:27:25.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.757+0000 7fdbd8cc8640 1 Processor -- start 2024-04-26T22:27:25.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 -- start start 2024-04-26T22:27:25.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbd4073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdbd40cffd0 0x7fdbd40d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd415d810 con 0x7fdbd406d250 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd415e400 con 0x7fdbd40cffd0 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.758+0000 7fdbd8cc8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd415dad0 con 0x7fdbd40b0d20 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd259c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fdbd406d250 0x7fdbd406d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46254/0 (socket says 172.21.15.62:46254) 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd259c640 1 -- 172.21.15.62:0/505386365 learned_addr learned my addr 172.21.15.62:0/505386365 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbd4073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/505386365 >> v2:172.21.15.174:3300/0 conn(0x7fdbd40cffd0 0x7fdbd40d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 854709191 0 0) 0x7fdbd415d810 con 0x7fdbd406d250 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbc0003520 con 0x7fdbd406d250 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3011208681 0 0) 0x7fdbc0003520 con 0x7fdbd406d250 2024-04-26T22:27:25.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 msgr2=0x7fdbd4073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbd4073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 >> v2:172.21.15.174:3300/0 conn(0x7fdbd40cffd0 msgr2=0x7fdbd40d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 --2- 172.21.15.62:0/505386365 >> v2:172.21.15.174:3300/0 conn(0x7fdbd40cffd0 0x7fdbd40d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd415ddb0 con 0x7fdbd406d250 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fdbc4001670 con 0x7fdbd406d250 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbd4073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 >> v1:172.21.15.62:6789/0 conn(0x7fdbd406d250 legacy=0x7fdbd406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbc00039b0 0x7fdbc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd415e400 con 0x7fdbc0004d40 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd415dad0 con 0x7fdbc00039b0 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc000b9c0 con 0x7fdbd40b0d20 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbd259c640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbc00039b0 0x7fdbc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.759+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 msgr2=0x7fdbc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd40b0d20 0x7fdbc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbc00039b0 msgr2=0x7fdbc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbc00039b0 0x7fdbc0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/505386365 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc0003520 con 0x7fdbc0004d40 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fc0 tx=0x7fdbc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca70eebe4ed39d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc009c10 con 0x7fdbc0004d40 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc036070 con 0x7fdbc0004d40 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.760+0000 7fdbd1d9b640 1 -- 172.21.15.62:0/505386365 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc02d070 con 0x7fdbc0004d40 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/505386365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 msgr2=0x7fdbc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/505386365 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbc00051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fc0 tx=0x7fdbc0005830 comp rx=0 tx=0).stop 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/505386365 reap_dead start 2024-04-26T22:27:25.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/505386365 shutdown_connections 2024-04-26T22:27:25.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/505386365 >> 172.21.15.62:0/505386365 conn(0x7fdbd4072880 msgr2=0x7fdbd4072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/505386365 shutdown_connections 2024-04-26T22:27:25.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.761+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/505386365 wait complete. 2024-04-26T22:27:25.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.762+0000 7fdbd8cc8640 1 Processor -- start 2024-04-26T22:27:25.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.762+0000 7fdbd8cc8640 1 -- start start 2024-04-26T22:27:25.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbd413f610 0x7fdbd40afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd413e270 0x7fdbd406c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4154770 con 0x7fdbc0004d40 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41545f0 con 0x7fdbd413f610 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd8cc8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd4073370 con 0x7fdbd413e270 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51298/0 (socket says 172.21.15.62:51298) 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 -- 172.21.15.62:0/1294392427 learned_addr learned my addr 172.21.15.62:0/1294392427 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbd413f610 0x7fdbd40afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd413e270 0x7fdbd406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 -- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd413e270 msgr2=0x7fdbd406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd413e270 0x7fdbd406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 -- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbd413f610 msgr2=0x7fdbd40afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbd413f610 0x7fdbd40afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.763+0000 7fdbd259c640 1 -- 172.21.15.62:0/1294392427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd413d1d0 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4007310 tx=0x7fdbc40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1294392427 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400f960 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1294392427 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4018070 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd413d4b0 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1294392427 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc400fd40 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.764+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd413da70 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.766+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1294392427 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdbc401d460 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.766+0000 7fdbcaffd640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 0x7fdbb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.766+0000 7fdbcaffd640 1 -- 172.21.15.62:0/1294392427 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbc408bb40 con 0x7fdbc0004d40 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.766+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 0x7fdbb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.767+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 0x7fdbb406ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdbd406cf90 tx=0x7fdbbc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.767+0000 7fdbdb9e3740 1 Processor -- start 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.767+0000 7fdbdb9e3740 1 -- start start 2024-04-26T22:27:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56eee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ea6120 0x55fb56eec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb56e0a740 con 0x55fb56ed0440 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb56da63a0 con 0x55fb56eedce0 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55fb56e06700 con 0x55fb56ea6120 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51300/0 (socket says 172.21.15.62:51300) 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 -- 172.21.15.62:0/817929733 learned_addr learned my addr 172.21.15.62:0/817929733 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 -- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ea6120 msgr2=0x55fb56eec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ea6120 0x55fb56eec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56eee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 -- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 msgr2=0x55fb56eee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56eee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 -- 172.21.15.62:0/817929733 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb56dc2750 con 0x55fb56ed0440 2024-04-26T22:27:25.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbd259c640 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4001d00 tx=0x7fdbc4059790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af4144a50f06abf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbb3fff640 1 -- 172.21.15.62:0/817929733 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4062450 con 0x55fb56ed0440 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbb3fff640 1 -- 172.21.15.62:0/817929733 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4057b90 con 0x55fb56ed0440 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.768+0000 7fdbb3fff640 1 -- 172.21.15.62:0/817929733 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4058ce0 con 0x55fb56ed0440 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/817929733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 msgr2=0x55fb56ed0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4001d00 tx=0x7fdbc4059790 comp rx=0 tx=0).stop 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/817929733 shutdown_connections 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ea6120 0x55fb56eec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56eee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/817929733 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ed0440 0x55fb56ed0820 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/817929733 >> 172.21.15.62:0/817929733 conn(0x55fb56ea2a50 msgr2=0x55fb56eeb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/817929733 shutdown_connections 2024-04-26T22:27:25.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.769+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/817929733 wait complete. 2024-04-26T22:27:25.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.770+0000 7fdbdb9e3740 1 -- start start 2024-04-26T22:27:25.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.770+0000 7fdbdb9e3740 1 Processor -- start 2024-04-26T22:27:25.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ecbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56ee3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb56ea6ec0 con 0x55fb56ee39d0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb56d91fe0 con 0x55fb56eedce0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55fb56ea0980 con 0x55fb56ed0440 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbdb9e3740 10 client.0 _ll_register_callbacks cb 0x55fb56da8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51316/0 (socket says 172.21.15.62:51316) 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/1461914913 learned_addr learned my addr 172.21.15.62:0/1461914913 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56ee3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ecbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 msgr2=0x55fb56ecbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ecbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 msgr2=0x55fb56ee3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56ee3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/1461914913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb56b1c2d0 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ecbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc007090 tx=0x7fdbcc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c01c11bfe904c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.771+0000 7fdbb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc031d60 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbcc003990 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc4057b90 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc04e050 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 10 client.4641 fetch_fsmap learned FSMap version 7 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55fb56be66b0 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc4057b90 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fdbcc051020 con 0x55fb56ee39d0 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T22:27:25.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 10 client.4641 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 20 client.4641 populate_metadata read hostname 'smithi062' 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56ed1300 0x55fb56efbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 4 client.4641 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 --> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb56e18350 con 0x55fb56ed1300 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56ed1300 0x55fb56efbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbb17fa640 10 client.4641 ms_handle_connect on v2:172.21.15.62:6833/3095900185 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.772+0000 7fdbd259c640 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56ed1300 0x55fb56efbdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89949ba2d0d3e7d server_cookie=bec2a4753e6e7b54 in_seq=0 out_seq=0 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbb17fa640 1 -- 172.21.15.62:0/1461914913 <== mds.0 v2:172.21.15.62:6833/3095900185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb56e18350 con 0x55fb56ed1300 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbb17fa640 10 client.4641 handle_command_reply: tid=0 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 client.4641 shutdown 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 msgr2=0x55fb56eeb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc007090 tx=0x7fdbcc009750 comp rx=0 tx=0).stop 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 shutdown_connections 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eeb280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56eedce0 0x55fb56ee3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56ed1300 0x55fb56efbdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/1461914913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ecbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 >> 172.21.15.62:0/1461914913 conn(0x55fb56ea6120 msgr2=0x55fb56eef060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbb17fa640 0 client.4641 ms_handle_reset on v2:172.21.15.62:6833/3095900185 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 shutdown_connections 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.773+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/1461914913 wait complete. 2024-04-26T22:27:25.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.774+0000 7fdbdb9e3740 20 client.4641 trim_cache size 0 max 16384 2024-04-26T22:27:25.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.792+0000 7fdbdb9e3740 1 Processor -- start 2024-04-26T22:27:25.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.792+0000 7fdbdb9e3740 1 -- start start 2024-04-26T22:27:25.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ea6120 0x55fb56e9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ea66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd41fec50 con 0x55fb56ee39d0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd4152730 con 0x55fb56ea6120 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd41e6260 con 0x55fb56ed0440 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51318/0 (socket says 172.21.15.62:51318) 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/3419380237 learned_addr learned my addr 172.21.15.62:0/3419380237 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 msgr2=0x55fb56ea66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ea66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ea6120 msgr2=0x55fb56e9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ea6120 0x55fb56e9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 -- 172.21.15.62:0/3419380237 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb56ea0d20 con 0x55fb56ee39d0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fd0 tx=0x7fdbcc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af64e704d565178c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbb37fe640 1 -- 172.21.15.62:0/3419380237 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc031b60 con 0x55fb56ee39d0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbb37fe640 1 -- 172.21.15.62:0/3419380237 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc036040 con 0x55fb56ee39d0 2024-04-26T22:27:25.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.793+0000 7fdbb37fe640 1 -- 172.21.15.62:0/3419380237 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc003f50 con 0x55fb56ee39d0 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/3419380237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 msgr2=0x55fb56eebf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009fd0 tx=0x7fdbcc040000 comp rx=0 tx=0).stop 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/3419380237 shutdown_connections 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ee39d0 0x55fb56eebf00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56ea66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/3419380237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ea6120 0x55fb56e9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/3419380237 >> 172.21.15.62:0/3419380237 conn(0x55fb56ed2480 msgr2=0x55fb56efa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/3419380237 shutdown_connections 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/3419380237 wait complete. 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 -- start start 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.794+0000 7fdbdb9e3740 1 Processor -- start 2024-04-26T22:27:25.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56f0dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 0x55fb56f0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55fb56ed1180 con 0x55fb56ea6120 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb56ed46b0 con 0x55fb56ee39d0 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55fb56ed17d0 con 0x55fb56ed0440 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbdb9e3740 10 client.0 _ll_register_callbacks cb 0x7fdbcc0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbd259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51322/0 (socket says 172.21.15.62:51322) 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.795+0000 7fdbd259c640 1 -- 172.21.15.62:0/993692189 learned_addr learned my addr 172.21.15.62:0/993692189 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56f0dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 0x55fb56f0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 -- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 msgr2=0x55fb56f0dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56f0dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 -- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 msgr2=0x55fb56f0e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 0x55fb56f0e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 -- 172.21.15.62:0/993692189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb56eed180 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56f0dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd2d9d640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 0x55fb56f0e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbd259c640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbc408a0c0 tx=0x7fdbc4089f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff9ad7701f3c147 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4055f00 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbc4058ce0 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc036040 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fdbc40560a0 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 10 client.4647 fetch_fsmap learned FSMap version 7 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55fb56ee40e0 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbcc036040 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7fdbc405ab10 con 0x55fb56ea6120 2024-04-26T22:27:25.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 10 client.4647 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 10 client.4647 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 20 client.4647 populate_metadata read hostname 'smithi062' 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56eedce0 0x55fb56efcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 4 client.4647 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.796+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 --> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] -- command(tid 0: {"prefix": "status"}) -- 0x55fb56e121c0 con 0x55fb56eedce0 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56eedce0 0x55fb56efcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbb17fa640 10 client.4647 ms_handle_connect on v2:172.21.15.62:6833/3095900185 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbcbfff640 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56eedce0 0x55fb56efcea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f637a74fa6db3db server_cookie=ac7d4434761f52cc in_seq=0 out_seq=0 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbb17fa640 1 -- 172.21.15.62:0/993692189 <== mds.0 v2:172.21.15.62:6833/3095900185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55fb56e121c0 con 0x55fb56eedce0 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbb17fa640 10 client.4647 handle_command_reply: tid=0 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 client.4647 shutdown 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 msgr2=0x55fb56ea43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbc408a0c0 tx=0x7fdbc4089f40 comp rx=0 tx=0).stop 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 shutdown_connections 2024-04-26T22:27:25.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb56ee39d0 0x55fb56f0e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:6833/3095900185,v1:172.21.15.62:6835/3095900185] conn(0x55fb56eedce0 0x55fb56efcea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55fb56ed0440 0x55fb56f0dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 --2- 172.21.15.62:0/993692189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55fb56ea6120 0x55fb56ea43f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 >> 172.21.15.62:0/993692189 conn(0x55fb56ed2480 msgr2=0x55fb56efa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbb17fa640 0 client.4647 ms_handle_reset on v2:172.21.15.62:6833/3095900185 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 shutdown_connections 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 1 -- 172.21.15.62:0/993692189 wait complete. 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.797+0000 7fdbdb9e3740 20 client.4647 trim_cache size 0 max 16384 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "f70b0a7c-519a-45f1-aa13-a1f2de1080a4", 2024-04-26T22:27:25.803 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4287, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 11.927473424, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 7, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 20, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 20, 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 18.590633516 2024-04-26T22:27:25.804 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 msgr2=0x7fdbb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 0x7fdbb406ede0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdbd406cf90 tx=0x7fdbbc009190 comp rx=0 tx=0).stop 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 msgr2=0x7fdbd413cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4007310 tx=0x7fdbc40077d0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 shutdown_connections 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdbb406c920 0x7fdbb406ede0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdbd413e270 0x7fdbd406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbd413f610 0x7fdbd40afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 --2- 172.21.15.62:0/1294392427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdbc0004d40 0x7fdbd413cb30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:25.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 >> 172.21.15.62:0/1294392427 conn(0x7fdbd4072880 msgr2=0x7fdbd40b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:25.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 shutdown_connections 2024-04-26T22:27:25.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:25.798+0000 7fdbd8cc8640 1 -- 172.21.15.62:0/1294392427 wait complete. 2024-04-26T22:27:25.809 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T22:27:25.818 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T22:27:25.818 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T22:27:25.819 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T22:27:25.819 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T22:27:25.925 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T22:27:25.925 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T22:27:25.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:25.925 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:27:25.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:25.941 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-26T22:27:26.006 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-26T22:27:26.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:27:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.009 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:27:26.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link delete ceph-brx 2024-04-26T22:27:26.130 INFO:teuthology.orchestra.run.smithi062.stderr:Cannot find device "ceph-brx" 2024-04-26T22:27:26.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:27:26.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.132 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:27:26.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2024-04-26T22:27:26.222 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2024-04-26T22:27:26.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:27:26.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.223 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:27:26.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.281 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2024-04-26T22:27:26.349 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2024-04-26T22:27:26.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T22:27:26.351 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T22:27:26.351 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:27:26.351 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:27:26.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:27:26.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.452+0000 7f5623f0a640 1 Processor -- start 2024-04-26T22:27:26.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.452+0000 7f5623f0a640 1 -- start start 2024-04-26T22:27:26.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5623f0a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 0x7f561c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5623f0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5623f0a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f561c0b1d70 con 0x7f561c0b2050 2024-04-26T22:27:26.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5623f0a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f561c163220 con 0x7f561c162e40 2024-04-26T22:27:26.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5623f0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f561c0d0eb0 con 0x7f561c0cff40 2024-04-26T22:27:26.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f56224a7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f56214a5640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f561c0b2050 0x7f561c0cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:48532/0 (socket says 172.21.15.62:48532) 2024-04-26T22:27:26.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f5621ca6640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 0x7f561c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.453+0000 7f56214a5640 1 -- 172.21.15.62:0/2917234349 learned_addr learned my addr 172.21.15.62:0/2917234349 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f5620ca4640 1 -- 172.21.15.62:0/2917234349 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829546272 0 0) 0x7f561c0b1d70 con 0x7f561c0b2050 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f5620ca4640 1 -- 172.21.15.62:0/2917234349 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5608003520 con 0x7f561c0b2050 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f56224a7640 1 -- 172.21.15.62:0/2917234349 >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 msgr2=0x7f561c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f56224a7640 1 --2- 172.21.15.62:0/2917234349 >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 0x7f561c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f56224a7640 1 -- 172.21.15.62:0/2917234349 >> v1:172.21.15.174:6789/0 conn(0x7f561c0b2050 legacy=0x7f561c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f56224a7640 1 -- 172.21.15.62:0/2917234349 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c0d1960 con 0x7f561c0cff40 2024-04-26T22:27:26.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.454+0000 7f5621ca6640 1 --2- 172.21.15.62:0/2917234349 >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 0x7f561c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:26.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f56224a7640 1 --2- 172.21.15.62:0/2917234349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0ce010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5614009d90 tx=0x7f561402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5bb5293636a20bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:26.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5620ca4640 1 -- 172.21.15.62:0/2917234349 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614031600 con 0x7f561c0cff40 2024-04-26T22:27:26.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5620ca4640 1 -- 172.21.15.62:0/2917234349 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56140317a0 con 0x7f561c0cff40 2024-04-26T22:27:26.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5620ca4640 1 -- 172.21.15.62:0/2917234349 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614031b50 con 0x7f561c0cff40 2024-04-26T22:27:26.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 -- 172.21.15.62:0/2917234349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 msgr2=0x7f561c0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2917234349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0ce010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5614009d90 tx=0x7f561402a5d0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 -- 172.21.15.62:0/2917234349 shutdown_connections 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2917234349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0ce010 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2917234349 >> v2:172.21.15.174:3300/0 conn(0x7f561c162e40 0x7f561c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 -- 172.21.15.62:0/2917234349 >> 172.21.15.62:0/2917234349 conn(0x7f561c09ba50 msgr2=0x7f561c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.455+0000 7f5623f0a640 1 -- 172.21.15.62:0/2917234349 shutdown_connections 2024-04-26T22:27:26.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.456+0000 7f5623f0a640 1 -- 172.21.15.62:0/2917234349 wait complete. 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 Processor -- start 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 -- start start 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 0x7f561c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f561c0b24e0 con 0x7f561c162e40 2024-04-26T22:27:26.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f561c0d1f20 con 0x7f561c0b2050 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.457+0000 7f5623f0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f561c06ea40 con 0x7f561c0cff40 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51336/0 (socket says 172.21.15.62:51336) 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 -- 172.21.15.62:0/2448514276 learned_addr learned my addr 172.21.15.62:0/2448514276 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f5621ca6640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 0x7f561c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56214a5640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 -- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 msgr2=0x7f561c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 -- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 msgr2=0x7f561c0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 0x7f561c0b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 -- 172.21.15.62:0/2448514276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f561c0b5e30 con 0x7f561c162e40 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56214a5640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f5621ca6640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 0x7f561c0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:26.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56224a7640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5614009eb0 tx=0x7f5614004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614040070 con 0x7f561c162e40 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561403c850 con 0x7f561c162e40 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f561c1f50d0 con 0x7f561c162e40 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561403cc40 con 0x7f561c162e40 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.458+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f561c1f5690 con 0x7f561c162e40 2024-04-26T22:27:26.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.459+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561c06a360 con 0x7f561c162e40 2024-04-26T22:27:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.460+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5614039040 con 0x7f561c162e40 2024-04-26T22:27:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.460+0000 7f56127fc640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.460+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56140acdb0 con 0x7f561c162e40 2024-04-26T22:27:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.460+0000 7f5621ca6640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:26.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.461+0000 7f5621ca6640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 0x7f560006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5618009af0 tx=0x7f5618008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:26.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.464+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5614038350 con 0x7f561c162e40 2024-04-26T22:27:26.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.688+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f561c0ceff0 con 0x7f561c162e40 2024-04-26T22:27:26.688 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.688+0000 7f56127fc640 1 -- 172.21.15.62:0/2448514276 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f561407b1e0 con 0x7f561c162e40 2024-04-26T22:27:26.689 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 msgr2=0x7f560006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 0x7f560006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5618009af0 tx=0x7f5618008040 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 msgr2=0x7f561c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5614009eb0 tx=0x7f5614004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 shutdown_connections 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f560006c8f0 0x7f560006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f561c162e40 0x7f561c0b3270 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f561c0cff40 0x7f561c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 --2- 172.21.15.62:0/2448514276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f561c0b2050 0x7f561c0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:26.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 >> 172.21.15.62:0/2448514276 conn(0x7f561c09ba50 msgr2=0x7f561c0ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:26.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 shutdown_connections 2024-04-26T22:27:26.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26.690+0000 7f5623f0a640 1 -- 172.21.15.62:0/2448514276 wait complete. 2024-04-26T22:27:26.701 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:27:26.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:27:26.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-26T22:27:26.701 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:27:26.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:27:26.702 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:27:26.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:27:26.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:27:26.702 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:27:26.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.702 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-26T22:27:26.717 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:26.718 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42747sec preferred_lft 42747sec 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6c94:f27c:3545:34e3/64 scope link noprefixroute 2024-04-26T22:27:26.719 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:26.719 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:27:26.719 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr flush dev ceph-brx 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set ceph-brx up 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:27:26.720 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:26.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:26.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:26.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.897 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:27:26.922 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-26T22:27:26.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:26.924 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-26T22:27:26.996 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-26T22:27:26.996 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:27:26.996 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:27:26.996 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:27:26.996 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:27:26.997 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:27.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:27.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:27.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:27.277 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:27:27.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:27.292 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-04-26T22:27:27.347 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:27.347 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:27.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:27:27.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T22:27:27.347 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:27.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:27.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:27.467 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T22:27:27.467 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:27.467 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:27.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T22:27:27.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:27:27.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T22:27:27.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T22:27:27.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T22:27:27.468 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:27.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:27.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:27.597 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:27.598 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:27.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-26T22:27:27.598 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:27:27.598 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:27.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:27.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:27.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:27.677 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:27:27.692 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:27:27.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:27.692 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:27:27.746 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:27:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:27.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:27:27.841 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:27:27.841 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:27:27.841 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:27.953 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:27:27.953 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:27:27.953 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:27.953 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:27.953 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:27:27.958 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:27:27.959 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:27:27.961 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:27:27.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:27.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:27:27.987 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:28.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:28.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:28.089 INFO:teuthology.orchestra.run.smithi062.stdout:client4653 2024-04-26T22:27:28.089 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:28.089 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:28.105 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:28.196 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:28.201 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:28.207 INFO:teuthology.orchestra.run.smithi062.stdout:client4653 2024-04-26T22:27:28.207 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:28.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:28.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:28.268 INFO:teuthology.orchestra.run.smithi062.stdout:client4653 2024-04-26T22:27:28.268 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f70b0a7c-519a-45f1-aa13-a1f2de1080a4.client4653/status') 2024-04-26T22:27:28.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:28.331 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4653 (3)192.168.0.1:0/3030615467 2024-04-26T22:27:28.332 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-04-26T22:27:28.332 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-04-26T22:27:28.332 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-04-26T22:27:28.332 INFO:teuthology.orchestra.run.smithi062.stderr:69 bytes copied, 2.6779e-05 s, 2.6 MB/s 2024-04-26T22:27:28.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:28.339 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-26T22:27:28.339 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:27:28.339 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T22:27:28.339 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:27:28.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.439+0000 7fefac719640 1 Processor -- start 2024-04-26T22:27:28.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.439+0000 7fefac719640 1 -- start start 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefac719640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fefa40d3820 0x7fefa40d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefac719640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 0x7fefa40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefac719640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefa40d0210 con 0x7fefa4162b40 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefac719640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fefa40d41c0 con 0x7fefa40d3820 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefac719640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fefa40d04f0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefa9cb4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fefa40d3820 0x7fefa40d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefaa4b5640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fefa4162b40 0x7fefa40d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46260/0 (socket says 172.21.15.62:46260) 2024-04-26T22:27:28.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.440+0000 7fefaa4b5640 1 -- 172.21.15.62:0/2220890176 learned_addr learned my addr 172.21.15.62:0/2220890176 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefaacb6640 1 --2- 172.21.15.62:0/2220890176 >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 0x7fefa40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefa9cb4640 1 -- 172.21.15.62:0/2220890176 >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 msgr2=0x7fefa40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefa9cb4640 1 --2- 172.21.15.62:0/2220890176 >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 0x7fefa40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefa9cb4640 1 -- 172.21.15.62:0/2220890176 >> v1:172.21.15.62:6789/0 conn(0x7fefa4162b40 legacy=0x7fefa40d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefa9cb4640 1 -- 172.21.15.62:0/2220890176 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa40d07d0 con 0x7fefa40d3820 2024-04-26T22:27:28.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefaacb6640 1 --2- 172.21.15.62:0/2220890176 >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 0x7fefa40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.441+0000 7fefa9cb4640 1 --2- 172.21.15.62:0/2220890176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40d3820 0x7fefa40d3c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefa409ec60 tx=0x7fef9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c46a24911784d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefa94b3640 1 -- 172.21.15.62:0/2220890176 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98037070 con 0x7fefa40d3820 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefa94b3640 1 -- 172.21.15.62:0/2220890176 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef98032440 con 0x7fefa40d3820 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefa94b3640 1 -- 172.21.15.62:0/2220890176 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98032810 con 0x7fefa40d3820 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 -- 172.21.15.62:0/2220890176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40d3820 msgr2=0x7fefa40d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 --2- 172.21.15.62:0/2220890176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40d3820 0x7fefa40d3c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fefa409ec60 tx=0x7fef9802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 -- 172.21.15.62:0/2220890176 shutdown_connections 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 --2- 172.21.15.62:0/2220890176 >> v2:172.21.15.174:3300/0 conn(0x7fefa40b4ba0 0x7fefa40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 --2- 172.21.15.62:0/2220890176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40d3820 0x7fefa40d3c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 -- 172.21.15.62:0/2220890176 >> 172.21.15.62:0/2220890176 conn(0x7fefa409e550 msgr2=0x7fefa40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 -- 172.21.15.62:0/2220890176 shutdown_connections 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.442+0000 7fefac719640 1 -- 172.21.15.62:0/2220890176 wait complete. 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.443+0000 7fefac719640 1 Processor -- start 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.443+0000 7fefac719640 1 -- start start 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefa40d3820 0x7fefa4150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa4162b40 0x7fefa40ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefa40d12b0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fefa40d0d90 con 0x7fefa4162b40 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefac719640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fefa40cfe30 con 0x7fefa40d3820 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefaa4b5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefaa4b5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51348/0 (socket says 172.21.15.62:51348) 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefaa4b5640 1 -- 172.21.15.62:0/3092829210 learned_addr learned my addr 172.21.15.62:0/3092829210 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:28.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefaacb6640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa4162b40 0x7fefa40ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.444+0000 7fefa9cb4640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefa40d3820 0x7fefa4150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 -- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefa40d3820 msgr2=0x7fefa4150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefa40d3820 0x7fefa4150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 -- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa4162b40 msgr2=0x7fefa40ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa4162b40 0x7fefa40ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 -- 172.21.15.62:0/3092829210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa40caa90 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefaa4b5640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c430 tx=0x7fef9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c017070 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c005ce0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa40ccf70 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0072f0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.445+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa40cd530 con 0x7fefa40b4ba0 2024-04-26T22:27:28.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.446+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa406a360 con 0x7fefa40b4ba0 2024-04-26T22:27:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.446+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef9c01a070 con 0x7fefa40b4ba0 2024-04-26T22:27:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.447+0000 7fefa2ffd640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.447+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fef9c08aec0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.447+0000 7fefa9cb4640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:28.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.447+0000 7fefa9cb4640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef98034000 tx=0x7fef9802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:28.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.451+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef9c058ca0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.676+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fefa40d2960 con 0x7fefa40b4ba0 2024-04-26T22:27:28.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.677+0000 7fefa2ffd640 1 -- 172.21.15.62:0/3092829210 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fef9c05cfa0 con 0x7fefa40b4ba0 2024-04-26T22:27:28.677 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:27:28.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 msgr2=0x7fef9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef98034000 tx=0x7fef9802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:28.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 msgr2=0x7fefa40caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c430 tx=0x7fef9c00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 shutdown_connections 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefa4162b40 0x7fefa40ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefa40d3820 0x7fefa4150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 --2- 172.21.15.62:0/3092829210 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefa40b4ba0 0x7fefa40caf70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 >> 172.21.15.62:0/3092829210 conn(0x7fefa409e550 msgr2=0x7fefa40d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 shutdown_connections 2024-04-26T22:27:28.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28.679+0000 7fefac719640 1 -- 172.21.15.62:0/3092829210 wait complete. 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:27:28.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:27:28.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:27:28.691 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T22:27:28.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:28.693 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:28.709 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6c94:f27c:3545:34e3/64 scope link noprefixroute 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d2:58:c7:f5:1b:26 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:28.710 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2a:a6ff:fef2:1980/64 scope link 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d2:58:c7:f5:1b:26 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d058:c7ff:fef5:1b26/64 scope link tentative 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:28.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:28.711 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:27:28.772 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-26T22:27:28.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:28.774 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-26T22:27:28.830 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-26T22:27:28.831 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:27:28.831 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:27:28.831 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:27:28.831 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:27:28.831 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:28.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:28.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:28.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:28.958 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:27:28.973 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:27:28.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:28.974 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list-id 2024-04-26T22:27:29.029 INFO:teuthology.orchestra.run.smithi062.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T22:27:29.030 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:29.030 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:29.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T22:27:29.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T22:27:29.030 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:29.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:29.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:29.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether be:a9:79:87:50:55 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T22:27:29.171 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T22:27:29.172 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.172 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::bca9:79ff:fe87:5055/64 scope link 2024-04-26T22:27:29.172 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:29.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:29.237 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether be:a9:79:87:50:55 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::bca9:79ff:fe87:5055/64 scope link 2024-04-26T22:27:29.238 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:29.239 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T22:27:29.240 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:29.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:29.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.390 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:29.390 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:27:29.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.1 up 2024-04-26T22:27:29.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T22:27:29.391 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:27:29.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:29.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:29.467 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T22:27:29.482 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T22:27:29.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:29.482 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T22:27:29.537 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:27:29.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:29.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T22:27:29.619 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T22:27:29.620 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T22:27:29.620 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:29.628 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T22:27:29.628 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T22:27:29.628 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:29.628 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:29.628 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:27:29.634 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:27:29.637 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T22:27:29.637 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:27:29.637 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:27:29.637 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:27:29.638 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:27:29.638 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:27:29.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:29.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T22:27:29.709 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:29.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:29.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.806 INFO:teuthology.orchestra.run.smithi062.stdout:client4513 2024-04-26T22:27:29.806 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:29.807 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:29.823 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:29.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:29.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.922 INFO:teuthology.orchestra.run.smithi062.stdout:client4513 2024-04-26T22:27:29.923 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:29.971 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:29.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:29.978 INFO:teuthology.orchestra.run.smithi062.stdout:client4513 2024-04-26T22:27:29.979 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f70b0a7c-519a-45f1-aa13-a1f2de1080a4.client4513/status') 2024-04-26T22:27:30.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:30.037 INFO:teuthology.orchestra.run.smithi062.stdout:instance: client.4513 (3)172.21.15.62:0/1509191311 2024-04-26T22:27:30.037 INFO:teuthology.orchestra.run.smithi062.stdout:blocklisted: false 2024-04-26T22:27:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records in 2024-04-26T22:27:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:0+1 records out 2024-04-26T22:27:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:70 bytes copied, 2.8354e-05 s, 2.5 MB/s 2024-04-26T22:27:30.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:30.044 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T22:27:30.045 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:27:30.045 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:27:30.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:27:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.144+0000 7f529b66d640 1 Processor -- start 2024-04-26T22:27:30.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.144+0000 7f529b66d640 1 -- start start 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f529b66d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5294073120 0x7f529406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f529b66d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529415be00 con 0x7f52940d0540 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f529b66d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529415c0a0 con 0x7f529406f2c0 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f529b66d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f529406e960 con 0x7f5294073120 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f5298c08640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5294073120 0x7f529406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f5299409640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f52940d0540 0x7f5294072a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:46270/0 (socket says 172.21.15.62:46270) 2024-04-26T22:27:30.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.145+0000 7f5299409640 1 -- 172.21.15.62:0/781788673 learned_addr learned my addr 172.21.15.62:0/781788673 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.146+0000 7f5298c08640 1 -- 172.21.15.62:0/781788673 >> v1:172.21.15.174:6789/0 conn(0x7f529406f2c0 legacy=0x7f529406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.146+0000 7f5298c08640 1 -- 172.21.15.62:0/781788673 >> v1:172.21.15.62:6789/0 conn(0x7f52940d0540 legacy=0x7f5294072a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.146+0000 7f5298c08640 1 -- 172.21.15.62:0/781788673 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529415c380 con 0x7f5294073120 2024-04-26T22:27:30.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.146+0000 7f5293fff640 1 -- 172.21.15.62:0/781788673 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749033453 0 0) 0x7f529415be00 con 0x7f52940d0540 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.146+0000 7f5298c08640 1 --2- 172.21.15.62:0/781788673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5294073120 0x7f529406ed80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52880097a0 tx=0x7f528802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3abc5dcb5eac2e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f5293fff640 1 -- 172.21.15.62:0/781788673 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288037070 con 0x7f5294073120 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f5293fff640 1 -- 172.21.15.62:0/781788673 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5288032440 con 0x7f5294073120 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f5293fff640 1 -- 172.21.15.62:0/781788673 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288032810 con 0x7f5294073120 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 -- 172.21.15.62:0/781788673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5294073120 msgr2=0x7f529406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 --2- 172.21.15.62:0/781788673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5294073120 0x7f529406ed80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52880097a0 tx=0x7f528802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 -- 172.21.15.62:0/781788673 shutdown_connections 2024-04-26T22:27:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 --2- 172.21.15.62:0/781788673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5294073120 0x7f529406ed80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 -- 172.21.15.62:0/781788673 >> 172.21.15.62:0/781788673 conn(0x7f52940d0d50 msgr2=0x7f52940d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 -- 172.21.15.62:0/781788673 shutdown_connections 2024-04-26T22:27:30.149 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.147+0000 7f529b66d640 1 -- 172.21.15.62:0/781788673 wait complete. 2024-04-26T22:27:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.148+0000 7f529b66d640 1 Processor -- start 2024-04-26T22:27:30.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 -- start start 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 0x7f529415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 0x7f529413b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f52940d0920 con 0x7f529406f2c0 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f529415c940 con 0x7f5294073120 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f529b66d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f52940d0aa0 con 0x7f52940d0540 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f5299409640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f5299409640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51362/0 (socket says 172.21.15.62:51362) 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.149+0000 7f5299409640 1 -- 172.21.15.62:0/205287435 learned_addr learned my addr 172.21.15.62:0/205287435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299c0a640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 0x7f529413b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5298c08640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 0x7f529415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:30.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 -- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 msgr2=0x7f529413b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 0x7f529413b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 -- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 msgr2=0x7f529415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 0x7f529415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 -- 172.21.15.62:0/205287435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529415e290 con 0x7f529406f2c0 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299c0a640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 0x7f529413b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5298c08640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 0x7f529415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5299409640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c430 tx=0x7f528c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0074e0 con 0x7f529406f2c0 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528c0047e0 con 0x7f529406f2c0 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5294151190 con 0x7f529406f2c0 2024-04-26T22:27:30.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c004bd0 con 0x7f529406f2c0 2024-04-26T22:27:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.150+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5294151750 con 0x7f529406f2c0 2024-04-26T22:27:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.151+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529406a360 con 0x7f529406f2c0 2024-04-26T22:27:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.152+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f528c011070 con 0x7f529406f2c0 2024-04-26T22:27:30.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.152+0000 7f5291ffb640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.152+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f528c08b780 con 0x7f529406f2c0 2024-04-26T22:27:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.152+0000 7f5298c08640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:30.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.153+0000 7f5298c08640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 0x7f528006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5288034000 tx=0x7f528802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:30.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.156+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f529415e290 con 0x7f529406f2c0 2024-04-26T22:27:30.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.380+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f529415d1d0 con 0x7f529406f2c0 2024-04-26T22:27:30.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.381+0000 7f5291ffb640 1 -- 172.21.15.62:0/205287435 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f528c059c40 con 0x7f529406f2c0 2024-04-26T22:27:30.381 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:27:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 msgr2=0x7f528006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 0x7f528006edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5288034000 tx=0x7f528802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 msgr2=0x7f5294150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:30.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c430 tx=0x7f528c00c8f0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 shutdown_connections 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f528006c8f0 0x7f528006edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f52940d0540 0x7f529413b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5294073120 0x7f529415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 --2- 172.21.15.62:0/205287435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f529406f2c0 0x7f5294150ab0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 >> 172.21.15.62:0/205287435 conn(0x7f52940d0d50 msgr2=0x7f52940ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 shutdown_connections 2024-04-26T22:27:30.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:30.383+0000 7f529b66d640 1 -- 172.21.15.62:0/205287435 wait complete. 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:27:30.394 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:27:30.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:30.394 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:30.413 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f0 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f1 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42728sec preferred_lft 42728sec 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::daa0:5f76:a60a:7388/64 scope link noprefixroute 2024-04-26T22:27:30.414 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:30.414 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add name ceph-brx type bridge 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr flush dev ceph-brx 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set ceph-brx up 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T22:27:30.415 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:30.509 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:30.604 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:30.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:30.611 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:27:30.635 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-04-26T22:27:30.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:30.637 DEBUG:teuthology.orchestra.run.smithi174:> route 2024-04-26T22:27:30.705 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2024-04-26T22:27:30.705 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:27:30.705 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:27:30.705 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:27:30.705 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:27:30.706 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:30.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:30.958 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:30.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:30.964 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:27:30.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:30.979 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list-id 2024-04-26T22:27:31.034 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:31.035 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:31.035 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:27:31.035 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T22:27:31.035 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:31.123 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:31.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.151 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T22:27:31.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T22:27:31.152 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T22:27:31.152 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:31.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:31.289 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.294 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:31.295 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:31.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.0 up 2024-04-26T22:27:31.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T22:27:31.295 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:31.343 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:31.364 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:31.370 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T22:27:31.385 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:27:31.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:31.385 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:27:31.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:27:31.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:31.440 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:27:31.532 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:27:31.532 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:27:31.532 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:31.635 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:27:31.636 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:27:31.636 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:31.636 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:31.636 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:31.654 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:27:31.655 INFO:teuthology.orchestra.run.smithi174.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:27:31.656 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:27:31.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:31.657 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:27:31.680 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:31.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:31.770 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.776 INFO:teuthology.orchestra.run.smithi174.stdout:client4475 2024-04-26T22:27:31.776 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:31.776 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:31.791 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:31.878 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:31.880 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.887 INFO:teuthology.orchestra.run.smithi174.stdout:client4475 2024-04-26T22:27:31.887 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:31.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:31.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:31.942 INFO:teuthology.orchestra.run.smithi174.stdout:client4475 2024-04-26T22:27:31.943 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f70b0a7c-519a-45f1-aa13-a1f2de1080a4.client4475/status') 2024-04-26T22:27:31.990 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:31.999 INFO:teuthology.orchestra.run.smithi174.stdout:instance: client.4475 (3)192.168.0.1:0/2141170046 2024-04-26T22:27:31.999 INFO:teuthology.orchestra.run.smithi174.stdout:blocklisted: false 2024-04-26T22:27:32.000 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records in 2024-04-26T22:27:32.000 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records out 2024-04-26T22:27:32.000 INFO:teuthology.orchestra.run.smithi174.stderr:69 bytes copied, 2.1691e-05 s, 3.2 MB/s 2024-04-26T22:27:32.000 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:32.007 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-26T22:27:32.007 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T22:27:32.007 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T22:27:32.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:27:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.120+0000 7f5eb8ff7640 1 Processor -- start 2024-04-26T22:27:32.120 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.120+0000 7f5eb8ff7640 1 -- start start 2024-04-26T22:27:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.121+0000 7f5eb8ff7640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5eb40d22e0 0x7f5eb40b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.121+0000 7f5eb8ff7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f760 0x7f5eb40b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.121+0000 7f5eb8ff7640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb40d3de0 con 0x7f5eb415f380 2024-04-26T22:27:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.121+0000 7f5eb8ff7640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb40d43a0 con 0x7f5eb40d22e0 2024-04-26T22:27:32.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.121+0000 7f5eb8ff7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb40d40c0 con 0x7f5eb415f760 2024-04-26T22:27:32.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb1d9b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5eb40d22e0 0x7f5eb40b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb2d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f760 0x7f5eb40b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb259c640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f5eb415f380 0x7f5eb40d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:48550/0 (socket says 172.21.15.62:48550) 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb259c640 1 -- 172.21.15.62:0/2490814345 learned_addr learned my addr 172.21.15.62:0/2490814345 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb1d9b640 1 -- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f760 msgr2=0x7f5eb40b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb1d9b640 1 --2- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f760 0x7f5eb40b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb1d9b640 1 -- 172.21.15.62:0/2490814345 >> v1:172.21.15.174:6789/0 conn(0x7f5eb415f380 legacy=0x7f5eb40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb1d9b640 1 -- 172.21.15.62:0/2490814345 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb40b8280 con 0x7f5eb40d22e0 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.122+0000 7f5eb159a640 1 -- 172.21.15.62:0/2490814345 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748777850 0 0) 0x7f5eb40d3de0 con 0x7f5eb415f380 2024-04-26T22:27:32.123 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb1d9b640 1 --2- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb40b8a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80097a0 tx=0x7f5ea802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec076daa07477562 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb159a640 1 -- 172.21.15.62:0/2490814345 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8037070 con 0x7f5eb40d22e0 2024-04-26T22:27:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb159a640 1 -- 172.21.15.62:0/2490814345 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea8032440 con 0x7f5eb40d22e0 2024-04-26T22:27:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb159a640 1 -- 172.21.15.62:0/2490814345 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8032810 con 0x7f5eb40d22e0 2024-04-26T22:27:32.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 msgr2=0x7f5eb40b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.123+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb40b8a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80097a0 tx=0x7f5ea802a210 comp rx=0 tx=0).stop 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/2490814345 shutdown_connections 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f760 0x7f5eb40b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/2490814345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb40b8a80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/2490814345 >> 172.21.15.62:0/2490814345 conn(0x7f5eb409e560 msgr2=0x7f5eb40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/2490814345 shutdown_connections 2024-04-26T22:27:32.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.124+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/2490814345 wait complete. 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.125+0000 7f5eb8ff7640 1 Processor -- start 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.125+0000 7f5eb8ff7640 1 -- start start 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f380 0x7f5eb4160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40d1280 con 0x7f5eb415f760 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb40d1100 con 0x7f5eb40d22e0 2024-04-26T22:27:32.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb8ff7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb40d5b50 con 0x7f5eb415f380 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51376/0 (socket says 172.21.15.62:51376) 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 -- 172.21.15.62:0/891033249 learned_addr learned my addr 172.21.15.62:0/891033249 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb259c640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb1d9b640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f380 0x7f5eb4160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 -- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f380 msgr2=0x7f5eb4160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f380 0x7f5eb4160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 -- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 msgr2=0x7f5eb415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 -- 172.21.15.62:0/891033249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb415c7f0 con 0x7f5eb415f760 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb259c640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.126+0000 7f5eb2d9d640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00efc0 tx=0x7f5eac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.127+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac018020 con 0x7f5eb415f760 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.127+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac014480 con 0x7f5eb415f760 2024-04-26T22:27:32.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.127+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb415c230 con 0x7f5eb415f760 2024-04-26T22:27:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.127+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac014870 con 0x7f5eb415f760 2024-04-26T22:27:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.127+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb415fd10 con 0x7f5eb415f760 2024-04-26T22:27:32.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.128+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb406a360 con 0x7f5eb415f760 2024-04-26T22:27:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.128+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5eac005d10 con 0x7f5eb415f760 2024-04-26T22:27:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.129+0000 7f5ea2ffd640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 0x7f5e9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.129+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5eac08c440 con 0x7f5eb415f760 2024-04-26T22:27:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.129+0000 7f5eb259c640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 0x7f5e9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:32.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.129+0000 7f5eb259c640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 0x7f5e9406ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f60 tx=0x7f5ea4007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:32.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.133+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5eac022070 con 0x7f5eb415f760 2024-04-26T22:27:32.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.358+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5eb40ba310 con 0x7f5eb415f760 2024-04-26T22:27:32.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.359+0000 7f5ea2ffd640 1 -- 172.21.15.62:0/891033249 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f5eac010040 con 0x7f5eb415f760 2024-04-26T22:27:32.359 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:27:32.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 msgr2=0x7f5e9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 0x7f5e9406ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4000f60 tx=0x7f5ea4007d50 comp rx=0 tx=0).stop 2024-04-26T22:27:32.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 msgr2=0x7f5eb40b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:32.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00efc0 tx=0x7f5eac00c490 comp rx=0 tx=0).stop 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 shutdown_connections 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e9406c920 0x7f5e9406ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5eb415f760 0x7f5eb40b6d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5eb415f380 0x7f5eb4160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 --2- 172.21.15.62:0/891033249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5eb40d22e0 0x7f5eb415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 >> 172.21.15.62:0/891033249 conn(0x7f5eb409e560 msgr2=0x7f5eb406e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 shutdown_connections 2024-04-26T22:27:32.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:32.361+0000 7f5eb8ff7640 1 -- 172.21.15.62:0/891033249 wait complete. 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T22:27:32.372 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:27:32.373 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T22:27:32.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.373 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2024-04-26T22:27:32.388 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f0 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f1 2024-04-26T22:27:32.389 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42726sec preferred_lft 42726sec 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::daa0:5f76:a60a:7388/64 scope link noprefixroute 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether ea:9b:af:31:81:76 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::d03c:ebff:fec0:f023/64 scope link tentative 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.390 INFO:teuthology.orchestra.run.smithi174.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:27:32.391 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether ea:9b:af:31:81:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:27:32.391 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::e89b:afff:fe31:8176/64 scope link tentative 2024-04-26T22:27:32.391 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.391 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:27:32.452 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-04-26T22:27:32.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.453 DEBUG:teuthology.orchestra.run.smithi174:> route 2024-04-26T22:27:32.509 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2024-04-26T22:27:32.509 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:27:32.509 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:27:32.510 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:27:32.510 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:27:32.510 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:32.597 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:32.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:32.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.633 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:27:32.648 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T22:27:32.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.649 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list-id 2024-04-26T22:27:32.703 INFO:teuthology.orchestra.run.smithi174.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T22:27:32.703 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:32.703 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:32.704 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T22:27:32.704 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T22:27:32.704 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:32.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:32.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:32.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.818 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 36:3f:0c:0f:de:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T22:27:32.841 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T22:27:32.842 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.842 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::343f:cff:fe0f:de91/64 scope link tentative 2024-04-26T22:27:32.842 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:32.843 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T22:27:32.906 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:27:32.906 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 36:3f:0c:0f:de:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::343f:cff:fe0f:de91/64 scope link tentative 2024-04-26T22:27:32.907 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:27:32.908 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T22:27:32.909 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:32.996 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:33.049 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.055 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:27:33.055 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:27:33.055 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.1 up 2024-04-26T22:27:33.055 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T22:27:33.055 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:27:33.102 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:33.124 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:33.130 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T22:27:33.145 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T22:27:33.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:27:33.145 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T22:27:33.199 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:27:33.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:33.200 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T22:27:33.280 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T22:27:33.280 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T22:27:33.280 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:33.288 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T22:27:33.288 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T22:27:33.289 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:33.289 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:33.289 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:27:33.307 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:27:33.308 INFO:teuthology.orchestra.run.smithi174.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T22:27:33.308 INFO:teuthology.orchestra.run.smithi174.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:27:33.308 INFO:teuthology.orchestra.run.smithi174.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:27:33.308 INFO:teuthology.orchestra.run.smithi174.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:27:33.308 INFO:teuthology.orchestra.run.smithi174.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:27:33.309 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:27:33.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:33.309 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T22:27:33.373 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:33.461 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:33.463 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.469 INFO:teuthology.orchestra.run.smithi174.stdout:client4528 2024-04-26T22:27:33.469 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:33.470 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:33.485 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:33.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:33.575 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.581 INFO:teuthology.orchestra.run.smithi174.stdout:client4528 2024-04-26T22:27:33.581 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:27:33.629 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:27:33.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.636 INFO:teuthology.orchestra.run.smithi174.stdout:client4528 2024-04-26T22:27:33.636 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f70b0a7c-519a-45f1-aa13-a1f2de1080a4.client4528/status') 2024-04-26T22:27:33.683 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:27:33.692 INFO:teuthology.orchestra.run.smithi174.stdout:instance: client.4528 (3)192.168.0.2:0/292299316 2024-04-26T22:27:33.692 INFO:teuthology.orchestra.run.smithi174.stdout:blocklisted: false 2024-04-26T22:27:33.693 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records in 2024-04-26T22:27:33.693 INFO:teuthology.orchestra.run.smithi174.stderr:0+1 records out 2024-04-26T22:27:33.693 INFO:teuthology.orchestra.run.smithi174.stderr:68 bytes copied, 2.1414e-05 s, 3.2 MB/s 2024-04-26T22:27:33.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:27:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:27:33.699 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T22:27:33.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.829+0000 7f3f54e23640 1 Processor -- start 2024-04-26T22:27:33.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.829+0000 7f3f54e23640 1 -- start start 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f54e23640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f54e23640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f500d0210 con 0x7f3f500d3820 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f54e23640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f500d04f0 con 0x7f3f50162b40 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f54e23640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500d07d0 con 0x7f3f500d3c80 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f4dd9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3f500d3820 0x7f3f500b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39718/0 (socket says 172.21.15.62:39718) 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f4dd9b640 1 -- 172.21.15.62:0/1113431381 learned_addr learned my addr 172.21.15.62:0/1113431381 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.830+0000 7f3f4ed9d640 1 --2- 172.21.15.62:0/1113431381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507459813 0 0) 0x7f3f500d0210 con 0x7f3f500d3820 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f38003520 con 0x7f3f500d3820 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537713057 0 0) 0x7f3f500d04f0 con 0x7f3f50162b40 2024-04-26T22:27:33.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f500d0210 con 0x7f3f50162b40 2024-04-26T22:27:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4ed9d640 1 -- 172.21.15.62:0/1113431381 >> v1:172.21.15.174:6789/0 conn(0x7f3f50162b40 legacy=0x7f3f500d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4ed9d640 1 -- 172.21.15.62:0/1113431381 >> v1:172.21.15.62:6789/0 conn(0x7f3f500d3820 legacy=0x7f3f500b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:33.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.831+0000 7f3f4ed9d640 1 -- 172.21.15.62:0/1113431381 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500d0ab0 con 0x7f3f500d3c80 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f4ed9d640 1 --2- 172.21.15.62:0/1113431381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500b52b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009dc0 tx=0x7f3f4802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd84bc65b4ca7dab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48007740 con 0x7f3f500d3c80 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f480078e0 con 0x7f3f500d3c80 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f4d59a640 1 -- 172.21.15.62:0/1113431381 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f48007c90 con 0x7f3f500d3c80 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f54e23640 1 -- 172.21.15.62:0/1113431381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 msgr2=0x7f3f500b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f54e23640 1 --2- 172.21.15.62:0/1113431381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500b52b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009dc0 tx=0x7f3f4802a6d0 comp rx=0 tx=0).stop 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f54e23640 1 -- 172.21.15.62:0/1113431381 shutdown_connections 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f54e23640 1 --2- 172.21.15.62:0/1113431381 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500b52b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:33.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.832+0000 7f3f54e23640 1 -- 172.21.15.62:0/1113431381 >> 172.21.15.62:0/1113431381 conn(0x7f3f5009e550 msgr2=0x7f3f500d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.833+0000 7f3f54e23640 1 -- 172.21.15.62:0/1113431381 shutdown_connections 2024-04-26T22:27:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.833+0000 7f3f54e23640 1 -- 172.21.15.62:0/1113431381 wait complete. 2024-04-26T22:27:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.834+0000 7f3f54e23640 1 Processor -- start 2024-04-26T22:27:33.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.834+0000 7f3f54e23640 1 -- start start 2024-04-26T22:27:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f50162b40 0x7f3f500cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500d12b0 con 0x7f3f500d3820 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f500d4110 con 0x7f3f50162b40 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f54e23640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f500cfe30 con 0x7f3f500d3c80 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42082/0 (socket says 172.21.15.62:42082) 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 -- 172.21.15.62:0/2521179699 learned_addr learned my addr 172.21.15.62:0/2521179699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4dd9b640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 -- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 msgr2=0x7f3f500cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4ed9d640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f50162b40 0x7f3f500cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 -- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f50162b40 msgr2=0x7f3f500cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f50162b40 0x7f3f500cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 -- 172.21.15.62:0/2521179699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500ce6d0 con 0x7f3f500d3820 2024-04-26T22:27:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4dd9b640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f4e59c640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c500 tx=0x7f3f4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.835+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400052d0 con 0x7f3f500d3820 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.836+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f40005490 con 0x7f3f500d3820 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.836+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f500ccc80 con 0x7f3f500d3820 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.836+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40012870 con 0x7f3f500d3820 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.836+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f500cec20 con 0x7f3f500d3820 2024-04-26T22:27:33.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.837+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5006a360 con 0x7f3f500d3820 2024-04-26T22:27:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.837+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f40012a10 con 0x7f3f500d3820 2024-04-26T22:27:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.837+0000 7f3f3effd640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.838+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f4008b830 con 0x7f3f500d3820 2024-04-26T22:27:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.838+0000 7f3f4dd9b640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.838+0000 7f3f4dd9b640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 0x7f3f3006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f500b6490 tx=0x7f3f44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:33.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:33.842+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f40059c70 con 0x7f3f500d3820 2024-04-26T22:27:34.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.029+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f5006d610 con 0x7f3f500d3820 2024-04-26T22:27:34.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.030+0000 7f3f3effd640 1 -- 172.21.15.62:0/2521179699 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f3f40059570 con 0x7f3f500d3820 2024-04-26T22:27:34.030 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:34.030 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 msgr2=0x7f3f3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 0x7f3f3006edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3f500b6490 tx=0x7f3f44009040 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 msgr2=0x7f3f500cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c500 tx=0x7f3f4000c9c0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 shutdown_connections 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f3006c8f0 0x7f3f3006edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f50162b40 0x7f3f500cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f500d3c80 0x7f3f500cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 --2- 172.21.15.62:0/2521179699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f500d3820 0x7f3f500cade0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 >> 172.21.15.62:0/2521179699 conn(0x7f3f5009e550 msgr2=0x7f3f5006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 shutdown_connections 2024-04-26T22:27:34.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.032+0000 7f3f54e23640 1 -- 172.21.15.62:0/2521179699 wait complete. 2024-04-26T22:27:34.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:34.044 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:34.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.147+0000 7fd8db059640 1 Processor -- start 2024-04-26T22:27:34.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.147+0000 7fd8db059640 1 -- start start 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8db059640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd8d40b5510 0x7fd8d40d1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8db059640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8d40d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8db059640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d40c9cc0 con 0x7fd8d40b6820 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8db059640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d40b5d80 con 0x7fd8d40b5510 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8db059640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d40b5fd0 con 0x7fd8d40b58f0 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8d3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd8d40b6820 0x7fd8d40d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39732/0 (socket says 172.21.15.62:39732) 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.148+0000 7fd8d3fff640 1 -- 172.21.15.62:0/2879713338 learned_addr learned my addr 172.21.15.62:0/2879713338 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:34.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/2879713338 >> v2:172.21.15.174:3300/0 conn(0x7fd8d40b5510 0x7fd8d40d1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437124433 0 0) 0x7fd8d40c9cc0 con 0x7fd8d40b6820 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8c4003520 con 0x7fd8d40b6820 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d95f6640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8d40d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3526436847 0 0) 0x7fd8c4003520 con 0x7fd8d40b6820 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 msgr2=0x7fd8d40d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8d40d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 >> v2:172.21.15.174:3300/0 conn(0x7fd8d40b5510 msgr2=0x7fd8d40d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 --2- 172.21.15.62:0/2879713338 >> v2:172.21.15.174:3300/0 conn(0x7fd8d40b5510 0x7fd8d40d1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d40d5c80 con 0x7fd8d40b6820 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fd8c0001670 con 0x7fd8d40b6820 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 >> v1:172.21.15.62:6789/0 conn(0x7fd8d40b6820 legacy=0x7fd8d40d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c40039b0 0x7fd8c4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c4004800 0x7fd8c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40b5d80 con 0x7fd8c40039b0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40b5fd0 con 0x7fd8c4004800 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c400b9f0 con 0x7fd8d40b58f0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c40039b0 0x7fd8c4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d3fff640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c4004800 0x7fd8c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d95f6640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 -- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 msgr2=0x7fd8c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d40b58f0 0x7fd8c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 -- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c4004800 msgr2=0x7fd8c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c4004800 0x7fd8c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d8df5640 1 -- 172.21.15.62:0/2879713338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c4003520 con 0x7fd8c40039b0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.149+0000 7fd8d3fff640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c4004800 0x7fd8c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.150+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c40039b0 0x7fd8c4003df0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009ac0 tx=0x7fd8c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1a75ee3400262 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.150+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8036070 con 0x7fd8c40039b0 2024-04-26T22:27:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.150+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c8031400 con 0x7fd8c40039b0 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.150+0000 7fd8d37fe640 1 -- 172.21.15.62:0/2879713338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8009690 con 0x7fd8c40039b0 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 -- 172.21.15.62:0/2879713338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c40039b0 msgr2=0x7fd8c4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 --2- 172.21.15.62:0/2879713338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8c40039b0 0x7fd8c4003df0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009ac0 tx=0x7fd8c4005830 comp rx=0 tx=0).stop 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8d95f6640 1 -- 172.21.15.62:0/2879713338 reap_dead start 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 -- 172.21.15.62:0/2879713338 shutdown_connections 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 -- 172.21.15.62:0/2879713338 >> 172.21.15.62:0/2879713338 conn(0x7fd8d40d5160 msgr2=0x7fd8d40d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 -- 172.21.15.62:0/2879713338 shutdown_connections 2024-04-26T22:27:34.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.151+0000 7fd8db059640 1 -- 172.21.15.62:0/2879713338 wait complete. 2024-04-26T22:27:34.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.152+0000 7fd8db059640 1 Processor -- start 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.152+0000 7fd8db059640 1 -- start start 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c40039b0 0x7fd8d40b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d4162370 0x7fd8d4148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40d0b60 con 0x7fd8d4161ef0 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d40d0880 con 0x7fd8c40039b0 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8db059640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d40d4100 con 0x7fd8d4162370 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42104/0 (socket says 172.21.15.62:42104) 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 -- 172.21.15.62:0/643358984 learned_addr learned my addr 172.21.15.62:0/643358984 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c40039b0 0x7fd8d40b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d95f6640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d4162370 0x7fd8d4148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 -- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d4162370 msgr2=0x7fd8d4148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d4162370 0x7fd8d4148640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 -- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c40039b0 msgr2=0x7fd8d40b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c40039b0 0x7fd8d40b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.153+0000 7fd8d3fff640 1 -- 172.21.15.62:0/643358984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d41630a0 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8d3fff640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007340 tx=0x7fd8c0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000fa00 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c0018070 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d41601f0 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000fda0 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.154+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d41628a0 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.155+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d4160710 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.155+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8c001c410 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.155+0000 7fd8d17fa640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.156+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd8c008b890 con 0x7fd8d4161ef0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.156+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.156+0000 7fd8d8df5640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 0x7fd8b806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8033000 tx=0x7fd8c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.160+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8c0059670 con 0x7fd8d4161ef0 2024-04-26T22:27:34.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.345+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8d40693e0 con 0x7fd8d4161ef0 2024-04-26T22:27:34.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.345+0000 7fd8d17fa640 1 -- 172.21.15.62:0/643358984 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd8c005d970 con 0x7fd8d4161ef0 2024-04-26T22:27:34.345 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:34.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 msgr2=0x7fd8b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 0x7fd8b806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8033000 tx=0x7fd8c8008030 comp rx=0 tx=0).stop 2024-04-26T22:27:34.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 msgr2=0x7fd8d4148100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007340 tx=0x7fd8c0007800 comp rx=0 tx=0).stop 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 shutdown_connections 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd8b806c920 0x7fd8b806ede0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8d4162370 0x7fd8d4148640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8d4161ef0 0x7fd8d4148100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 --2- 172.21.15.62:0/643358984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c40039b0 0x7fd8d40b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 >> 172.21.15.62:0/643358984 conn(0x7fd8d40d5160 msgr2=0x7fd8d40d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 shutdown_connections 2024-04-26T22:27:34.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.347+0000 7fd8db059640 1 -- 172.21.15.62:0/643358984 wait complete. 2024-04-26T22:27:34.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:34.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.461+0000 7f98b6340640 1 Processor -- start 2024-04-26T22:27:34.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.461+0000 7f98b6340640 1 -- start start 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b6340640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98b015f470 0x7f98b00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b6340640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b00d2ed0 con 0x7f98b00d21b0 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b6340640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b00d31b0 con 0x7f98b00d0d30 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b6340640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f98b00d1190 con 0x7f98b015f470 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98affff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f98b015f470 0x7f98b00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b48dd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f98b00d21b0 0x7f98b00b7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39740/0 (socket says 172.21.15.62:39740) 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98b48dd640 1 -- 172.21.15.62:0/2663891471 learned_addr learned my addr 172.21.15.62:0/2663891471 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98affff640 1 -- 172.21.15.62:0/2663891471 >> v1:172.21.15.174:6789/0 conn(0x7f98b00d0d30 legacy=0x7f98b00d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98affff640 1 -- 172.21.15.62:0/2663891471 >> v1:172.21.15.62:6789/0 conn(0x7f98b00d21b0 legacy=0x7f98b00b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98affff640 1 -- 172.21.15.62:0/2663891471 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00d3490 con 0x7f98b015f470 2024-04-26T22:27:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.462+0000 7f98aeffd640 1 -- 172.21.15.62:0/2663891471 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173307307 0 0) 0x7f98b00d2ed0 con 0x7f98b00d21b0 2024-04-26T22:27:34.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.463+0000 7f98affff640 1 --2- 172.21.15.62:0/2663891471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b015f470 0x7f98b00d07f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98a00097a0 tx=0x7f98a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b660ba403747a069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.463+0000 7f98aeffd640 1 -- 172.21.15.62:0/2663891471 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0037070 con 0x7f98b015f470 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.463+0000 7f98aeffd640 1 -- 172.21.15.62:0/2663891471 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a0032440 con 0x7f98b015f470 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.463+0000 7f98aeffd640 1 -- 172.21.15.62:0/2663891471 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0032810 con 0x7f98b015f470 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 -- 172.21.15.62:0/2663891471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b015f470 msgr2=0x7f98b00d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 --2- 172.21.15.62:0/2663891471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b015f470 0x7f98b00d07f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98a00097a0 tx=0x7f98a002a230 comp rx=0 tx=0).stop 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 -- 172.21.15.62:0/2663891471 shutdown_connections 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 --2- 172.21.15.62:0/2663891471 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b015f470 0x7f98b00d07f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 -- 172.21.15.62:0/2663891471 >> 172.21.15.62:0/2663891471 conn(0x7f98b009e510 msgr2=0x7f98b00d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 -- 172.21.15.62:0/2663891471 shutdown_connections 2024-04-26T22:27:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.464+0000 7f98b6340640 1 -- 172.21.15.62:0/2663891471 wait complete. 2024-04-26T22:27:34.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.465+0000 7f98b6340640 1 Processor -- start 2024-04-26T22:27:34.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.465+0000 7f98b6340640 1 -- start start 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98b00d21b0 0x7f98b00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 0x7f98b00b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f98b015f850 con 0x7f98b00d0d30 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98b00d3a50 con 0x7f98b015f470 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b6340640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f98b00d3f70 con 0x7f98b00d21b0 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42120/0 (socket says 172.21.15.62:42120) 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 -- 172.21.15.62:0/2465033899 learned_addr learned my addr 172.21.15.62:0/2465033899 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b48dd640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 0x7f98b00b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 -- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98b00d21b0 msgr2=0x7f98b00b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98af7fe640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98b00d21b0 0x7f98b00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98b00d21b0 0x7f98b00b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 -- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 msgr2=0x7f98b00b97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 0x7f98b00b97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 -- 172.21.15.62:0/2465033899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00b9ed0 con 0x7f98b00d0d30 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98b48dd640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 0x7f98b00b97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.466+0000 7f98affff640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098c0 tx=0x7f98a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.467+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0037040 con 0x7f98b00d0d30 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.467+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a0030770 con 0x7f98b00d0d30 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.467+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b00ba1b0 con 0x7f98b00d0d30 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.467+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0030b40 con 0x7f98b00d0d30 2024-04-26T22:27:34.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.467+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b00b7fd0 con 0x7f98b00d0d30 2024-04-26T22:27:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.468+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b006e7b0 con 0x7f98b00d0d30 2024-04-26T22:27:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.468+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98a0044020 con 0x7f98b00d0d30 2024-04-26T22:27:34.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.468+0000 7f98acff9640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 0x7f989006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.469+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f98a00ac200 con 0x7f98b00d0d30 2024-04-26T22:27:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.469+0000 7f98af7fe640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 0x7f989006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.469+0000 7f98af7fe640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 0x7f989006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98b01f8b40 tx=0x7f9898009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.473+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98b006e7b0 con 0x7f98b00d0d30 2024-04-26T22:27:34.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.658+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98b006a360 con 0x7f98b00d0d30 2024-04-26T22:27:34.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.659+0000 7f98acff9640 1 -- 172.21.15.62:0/2465033899 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f98a0032e20 con 0x7f98b00d0d30 2024-04-26T22:27:34.659 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 msgr2=0x7f989006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 0x7f989006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98b01f8b40 tx=0x7f9898009040 comp rx=0 tx=0).stop 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 msgr2=0x7f98b0156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98a00098c0 tx=0x7f98a00046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 shutdown_connections 2024-04-26T22:27:34.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f989006c8f0 0x7f989006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98b015f470 0x7f98b00b97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98b00d21b0 0x7f98b00b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 --2- 172.21.15.62:0/2465033899 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f98b00d0d30 0x7f98b0156470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 >> 172.21.15.62:0/2465033899 conn(0x7f98b009e510 msgr2=0x7f98b0162ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 shutdown_connections 2024-04-26T22:27:34.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.661+0000 7f98b6340640 1 -- 172.21.15.62:0/2465033899 wait complete. 2024-04-26T22:27:34.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:34.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.775+0000 7f45c14bb640 1 Processor -- start 2024-04-26T22:27:34.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.775+0000 7f45c14bb640 1 -- start start 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45c14bb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45bc0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45c14bb640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc15d700 con 0x7f45bc15d320 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45c14bb640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0ce8f0 con 0x7f45bc0d1260 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45c14bb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0d09a0 con 0x7f45bc0d16f0 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45baffd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f45bc15d320 0x7f45bc0d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39754/0 (socket says 172.21.15.62:39754) 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45baffd640 1 -- 172.21.15.62:0/415170530 learned_addr learned my addr 172.21.15.62:0/415170530 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45bc0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784787021 0 0) 0x7f45bc15d700 con 0x7f45bc15d320 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.776+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45bc15d320 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687437345 0 0) 0x7f45bc0ce8f0 con 0x7f45bc0d1260 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45bc15d700 con 0x7f45bc0d1260 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1026515428 0 0) 0x7f45a8003520 con 0x7f45bc15d320 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 msgr2=0x7f45bc0b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45bc0b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 >> v1:172.21.15.174:6789/0 conn(0x7f45bc0d1260 legacy=0x7f45bc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0cebd0 con 0x7f45bc15d320 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f45b4001670 con 0x7f45bc15d320 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 >> v1:172.21.15.62:6789/0 conn(0x7f45bc15d320 legacy=0x7f45bc0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45a8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a8004120 0x7f45a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800bb50 con 0x7f45a8004d40 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800bd00 con 0x7f45a8004120 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800bf00 con 0x7f45bc0d16f0 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45baffd640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45a8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45ba7fc640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a8004120 0x7f45a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 -- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 msgr2=0x7f45a8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0d16f0 0x7f45a8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 -- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a8004120 msgr2=0x7f45a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a8004120 0x7f45a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45bb7fe640 1 -- 172.21.15.62:0/415170530 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003520 con 0x7f45a8004d40 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.777+0000 7f45ba7fc640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a8004120 0x7f45a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.778+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009fc0 tx=0x7f45a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaba31e1c1948a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.778+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac009c10 con 0x7f45a8004d40 2024-04-26T22:27:34.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.778+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac036070 con 0x7f45a8004d40 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.778+0000 7f45b9ffb640 1 -- 172.21.15.62:0/415170530 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac031880 con 0x7f45a8004d40 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 -- 172.21.15.62:0/415170530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a8004d40 msgr2=0x7f45a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 --2- 172.21.15.62:0/415170530 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45a8004d40 0x7f45a80051c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009fc0 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45bb7fe640 1 -- 172.21.15.62:0/415170530 reap_dead start 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 -- 172.21.15.62:0/415170530 shutdown_connections 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 -- 172.21.15.62:0/415170530 >> 172.21.15.62:0/415170530 conn(0x7f45bc0d2f80 msgr2=0x7f45bc0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 -- 172.21.15.62:0/415170530 shutdown_connections 2024-04-26T22:27:34.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.779+0000 7f45c14bb640 1 -- 172.21.15.62:0/415170530 wait complete. 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.780+0000 7f45c14bb640 1 Processor -- start 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.780+0000 7f45c14bb640 1 -- start start 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45bc06d350 0x7f45bc09d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 0x7f45bc0a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0b2cd0 con 0x7f45bc0c7820 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0ceeb0 con 0x7f45bc06d350 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45c14bb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0ce550 con 0x7f45bc0a0530 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42150/0 (socket says 172.21.15.62:42150) 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 -- 172.21.15.62:0/3238530321 learned_addr learned my addr 172.21.15.62:0/3238530321 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:34.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45baffd640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45bc06d350 0x7f45bc09d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 0x7f45bc0a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 -- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 msgr2=0x7f45bc0a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 0x7f45bc0a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 -- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45bc06d350 msgr2=0x7f45bc09d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45bc06d350 0x7f45bc09d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 -- 172.21.15.62:0/3238530321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc09f490 con 0x7f45bc0c7820 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45bb7fe640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 0x7f45bc0a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.781+0000 7f45ba7fc640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e7e0 tx=0x7f45a400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.782+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a40050d0 con 0x7f45bc0c7820 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.782+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4005270 con 0x7f45bc0c7820 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.782+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc09f770 con 0x7f45bc0c7820 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.782+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4005460 con 0x7f45bc0c7820 2024-04-26T22:27:34.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.782+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc09fd30 con 0x7f45bc0c7820 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.783+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc0693e0 con 0x7f45bc0c7820 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.783+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45a401a020 con 0x7f45bc0c7820 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.784+0000 7f45b37fe640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 0x7f459c06ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.784+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f45a408b2d0 con 0x7f45bc0c7820 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.784+0000 7f45baffd640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 0x7f459c06ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:34.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.784+0000 7f45baffd640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 0x7f459c06ef50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b4001eb0 tx=0x7f45b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:34.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.788+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45a40597c0 con 0x7f45bc0c7820 2024-04-26T22:27:34.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.974+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45bc0cd670 con 0x7f45bc0c7820 2024-04-26T22:27:34.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.974+0000 7f45b37fe640 1 -- 172.21.15.62:0/3238530321 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f45a40590c0 con 0x7f45bc0c7820 2024-04-26T22:27:34.974 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 msgr2=0x7f459c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 0x7f459c06ef50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b4001eb0 tx=0x7f45b4009040 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 msgr2=0x7f45bc0c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e7e0 tx=0x7f45a400eca0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 shutdown_connections 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f459c06ca90 0x7f459c06ef50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f45bc0a0530 0x7f45bc0a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f45bc0c7820 0x7f45bc0c7c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 --2- 172.21.15.62:0/3238530321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45bc06d350 0x7f45bc09d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 >> 172.21.15.62:0/3238530321 conn(0x7f45bc0d2f80 msgr2=0x7f45bc0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:34.977 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 shutdown_connections 2024-04-26T22:27:34.978 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:34.976+0000 7f45c14bb640 1 -- 172.21.15.62:0/3238530321 wait complete. 2024-04-26T22:27:34.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:35.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.087+0000 7f7aaa55f640 1 Processor -- start 2024-04-26T22:27:35.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.087+0000 7f7aaa55f640 1 -- start start 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aaa55f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7aa40d1c50 0x7f7aa40c7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aaa55f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aaa55f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa40ced60 con 0x7f7aa415d310 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aaa55f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa40c6ab0 con 0x7f7aa40d1c50 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aaa55f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa40cf040 con 0x7f7aa40ce860 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.088+0000 7f7aa8afc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:52902/0 (socket says 172.21.15.62:52902) 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 -- 172.21.15.62:0/2527401242 learned_addr learned my addr 172.21.15.62:0/2527401242 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa37fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7aa40d1c50 0x7f7aa40c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 -- 172.21.15.62:0/2527401242 >> v2:172.21.15.174:3300/0 conn(0x7f7aa40d1c50 msgr2=0x7f7aa40c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 --2- 172.21.15.62:0/2527401242 >> v2:172.21.15.174:3300/0 conn(0x7f7aa40d1c50 0x7f7aa40c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa2ffd640 1 -- 172.21.15.62:0/2527401242 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480013336 0 0) 0x7f7aa40ced60 con 0x7f7aa415d310 2024-04-26T22:27:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 -- 172.21.15.62:0/2527401242 >> v1:172.21.15.174:6789/0 conn(0x7f7aa415d310 legacy=0x7f7aa40d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.089+0000 7f7aa8afc640 1 -- 172.21.15.62:0/2527401242 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa40cf320 con 0x7f7aa40ce860 2024-04-26T22:27:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.090+0000 7f7aa8afc640 1 --2- 172.21.15.62:0/2527401242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009e50 tx=0x7f7a9c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53710d747891dd5d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.090+0000 7f7aa2ffd640 1 -- 172.21.15.62:0/2527401242 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c036070 con 0x7f7aa40ce860 2024-04-26T22:27:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.090+0000 7f7aa2ffd640 1 -- 172.21.15.62:0/2527401242 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9c031600 con 0x7f7aa40ce860 2024-04-26T22:27:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.090+0000 7f7aa2ffd640 1 -- 172.21.15.62:0/2527401242 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0319d0 con 0x7f7aa40ce860 2024-04-26T22:27:35.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 -- 172.21.15.62:0/2527401242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 msgr2=0x7f7aa40c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/2527401242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009e50 tx=0x7f7a9c02a710 comp rx=0 tx=0).stop 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 -- 172.21.15.62:0/2527401242 shutdown_connections 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/2527401242 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40ce860 0x7f7aa40c7740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/2527401242 >> v2:172.21.15.174:3300/0 conn(0x7f7aa40d1c50 0x7f7aa40c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 -- 172.21.15.62:0/2527401242 >> 172.21.15.62:0/2527401242 conn(0x7f7aa40b4a50 msgr2=0x7f7aa40b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 -- 172.21.15.62:0/2527401242 shutdown_connections 2024-04-26T22:27:35.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.091+0000 7f7aaa55f640 1 -- 172.21.15.62:0/2527401242 wait complete. 2024-04-26T22:27:35.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.092+0000 7f7aaa55f640 1 Processor -- start 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.092+0000 7f7aaa55f640 1 -- start start 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 0x7f7aa41f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa415d310 0x7f7aa415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40b5970 con 0x7f7aa40ce860 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa40b57f0 con 0x7f7aa415d310 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa415d6f0 con 0x7f7aa40d1c50 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42154/0 (socket says 172.21.15.62:42154) 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 -- 172.21.15.62:0/3244249031 learned_addr learned my addr 172.21.15.62:0/3244249031 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:35.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa8afc640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa415d310 0x7f7aa415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa37fe640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 0x7f7aa41f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 -- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 msgr2=0x7f7aa41f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 0x7f7aa41f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 -- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa415d310 msgr2=0x7f7aa415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa415d310 0x7f7aa415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 -- 172.21.15.62:0/3244249031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa41f6b40 con 0x7f7aa40ce860 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa37fe640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 0x7f7aa41f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa3fff640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c410 tx=0x7f7a9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94017070 con 0x7f7aa40ce860 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a940035f0 con 0x7f7aa40ce860 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa41f6e10 con 0x7f7aa40ce860 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a940039e0 con 0x7f7aa40ce860 2024-04-26T22:27:35.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.093+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa41f73d0 con 0x7f7aa40ce860 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.094+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa40693e0 con 0x7f7aa40ce860 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.095+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a9401a070 con 0x7f7aa40ce860 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.095+0000 7f7aa0ff9640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 0x7f7a9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.095+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7a9408aea0 con 0x7f7aa40ce860 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.095+0000 7f7aa37fe640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 0x7f7a9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.096+0000 7f7aa37fe640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 0x7f7a9006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40b46f0 tx=0x7f7a98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.100+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7a94058d10 con 0x7f7aa40ce860 2024-04-26T22:27:35.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.284+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7aa40d1320 con 0x7f7aa40ce860 2024-04-26T22:27:35.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.285+0000 7f7aa0ff9640 1 -- 172.21.15.62:0/3244249031 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f7a9405d010 con 0x7f7aa40ce860 2024-04-26T22:27:35.285 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:35.286 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 msgr2=0x7f7a9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 0x7f7a9006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40b46f0 tx=0x7f7a98008040 comp rx=0 tx=0).stop 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 msgr2=0x7f7aa40cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400c410 tx=0x7f7a9400c8d0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 shutdown_connections 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7a9006c8f0 0x7f7a9006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aa415d310 0x7f7aa415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7aa40d1c50 0x7f7aa41f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 --2- 172.21.15.62:0/3244249031 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7aa40ce860 0x7f7aa40cbb40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 >> 172.21.15.62:0/3244249031 conn(0x7f7aa40b4a50 msgr2=0x7f7aa4070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 shutdown_connections 2024-04-26T22:27:35.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.287+0000 7f7aaa55f640 1 -- 172.21.15.62:0/3244249031 wait complete. 2024-04-26T22:27:35.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:35.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.402+0000 7fabcc6ea640 1 Processor -- start 2024-04-26T22:27:35.403 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.402+0000 7fabcc6ea640 1 -- start start 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.403+0000 7fabcc6ea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.403+0000 7fabcc6ea640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabc406c240 0x7fabc40b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.403+0000 7fabcc6ea640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc406ca20 con 0x7fabc40b4aa0 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.403+0000 7fabcc6ea640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fabc4068780 con 0x7fabc406c240 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.403+0000 7fabcc6ea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabc40b4ed0 con 0x7fabc4160370 2024-04-26T22:27:35.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9c85640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabc406c240 0x7fabc40b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabcac87640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fabc40b4aa0 0x7fabc4162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39758/0 (socket says 172.21.15.62:39758) 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabcac87640 1 -- 172.21.15.62:0/2877707495 learned_addr learned my addr 172.21.15.62:0/2877707495 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabca486640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9c85640 1 -- 172.21.15.62:0/2877707495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 msgr2=0x7fabc415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9c85640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9c85640 1 -- 172.21.15.62:0/2877707495 >> v1:172.21.15.62:6789/0 conn(0x7fabc40b4aa0 legacy=0x7fabc4162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9484640 1 -- 172.21.15.62:0/2877707495 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291783740 0 0) 0x7fabc406ca20 con 0x7fabc40b4aa0 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabc9c85640 1 -- 172.21.15.62:0/2877707495 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc406cd00 con 0x7fabc406c240 2024-04-26T22:27:35.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.404+0000 7fabca486640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabc9c85640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc40b4560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097d0 tx=0x7fabc002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e0c93167efd65e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabc9484640 1 -- 172.21.15.62:0/2877707495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0037070 con 0x7fabc406c240 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabc9484640 1 -- 172.21.15.62:0/2877707495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc0032440 con 0x7fabc406c240 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabc9484640 1 -- 172.21.15.62:0/2877707495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0032810 con 0x7fabc406c240 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 -- 172.21.15.62:0/2877707495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 msgr2=0x7fabc40b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc40b4560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fabc00097d0 tx=0x7fabc002a1c0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 -- 172.21.15.62:0/2877707495 shutdown_connections 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc40b4560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/2877707495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 -- 172.21.15.62:0/2877707495 >> 172.21.15.62:0/2877707495 conn(0x7fabc40d16b0 msgr2=0x7fabc40d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 -- 172.21.15.62:0/2877707495 shutdown_connections 2024-04-26T22:27:35.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.405+0000 7fabcc6ea640 1 -- 172.21.15.62:0/2877707495 wait complete. 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.406+0000 7fabcc6ea640 1 Processor -- start 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 -- start start 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc40b4aa0 0x7fabc41331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc41311d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabc406d5a0 con 0x7fabc406c240 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabc406d2c0 con 0x7fabc40b4aa0 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabcc6ea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabc406dac0 con 0x7fabc4160370 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabca486640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabca486640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42176/0 (socket says 172.21.15.62:42176) 2024-04-26T22:27:35.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.407+0000 7fabca486640 1 -- 172.21.15.62:0/3454135405 learned_addr learned my addr 172.21.15.62:0/3454135405 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 -- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 msgr2=0x7fabc41311d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc41311d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 -- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc40b4aa0 msgr2=0x7fabc41331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc40b4aa0 0x7fabc41331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 -- 172.21.15.62:0/3454135405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc4130130 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabca486640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fabb800e730 tx=0x7fabb800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb800f040 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb8004230 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc4130410 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb800ee40 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.408+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41309d0 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.409+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc406a360 con 0x7fabc406c240 2024-04-26T22:27:35.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.409+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabb800c6d0 con 0x7fabc406c240 2024-04-26T22:27:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.410+0000 7fabb6ffd640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 0x7faba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.410+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabb808b880 con 0x7fabc406c240 2024-04-26T22:27:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.410+0000 7fabc9c85640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 0x7faba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.411+0000 7fabc9c85640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 0x7faba806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fabc0034000 tx=0x7fabc002c000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.414+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabb8017020 con 0x7fabc406c240 2024-04-26T22:27:35.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.597+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabc4142580 con 0x7fabc406c240 2024-04-26T22:27:35.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.597+0000 7fabb6ffd640 1 -- 172.21.15.62:0/3454135405 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fabb8059cb0 con 0x7fabc406c240 2024-04-26T22:27:35.598 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 msgr2=0x7faba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 0x7faba806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fabc0034000 tx=0x7fabc002c000 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 msgr2=0x7fabc415f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fabb800e730 tx=0x7fabb800ebf0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 shutdown_connections 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7faba806c8f0 0x7faba806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc4160370 0x7fabc41311d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc40b4aa0 0x7fabc41331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 --2- 172.21.15.62:0/3454135405 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc406c240 0x7fabc415f8f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 >> 172.21.15.62:0/3454135405 conn(0x7fabc40d16b0 msgr2=0x7fabc40727d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 shutdown_connections 2024-04-26T22:27:35.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.599+0000 7fabcc6ea640 1 -- 172.21.15.62:0/3454135405 wait complete. 2024-04-26T22:27:35.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:35.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.714+0000 7fb946522640 1 Processor -- start 2024-04-26T22:27:35.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.714+0000 7fb946522640 1 -- start start 2024-04-26T22:27:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb946522640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb946522640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb9400d21b0 0x7fb9400b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb946522640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9400d2ed0 con 0x7fb9400d0d30 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb946522640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb9400d1190 con 0x7fb9400d21b0 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb946522640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb9400d31b0 con 0x7fb94015f470 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb93ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb944abf640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb9400d21b0 0x7fb9400b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb93f7fe640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb9400d0d30 0x7fb9400d1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:36786/0 (socket says 172.21.15.62:36786) 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb93f7fe640 1 -- 172.21.15.62:0/1779110332 learned_addr learned my addr 172.21.15.62:0/1779110332 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb944abf640 1 -- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 msgr2=0x7fb9400d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb944abf640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb944abf640 1 -- 172.21.15.62:0/1779110332 >> v1:172.21.15.174:6789/0 conn(0x7fb9400d0d30 legacy=0x7fb9400d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb944abf640 1 -- 172.21.15.62:0/1779110332 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400d3490 con 0x7fb9400d21b0 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb93effd640 1 -- 172.21.15.62:0/1779110332 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481005255 0 0) 0x7fb9400d2ed0 con 0x7fb9400d0d30 2024-04-26T22:27:35.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.715+0000 7fb93ffff640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.716+0000 7fb944abf640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d21b0 0x7fb9400b7fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb938009d90 tx=0x7fb93802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87a2ad0773c6fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.716+0000 7fb93effd640 1 -- 172.21.15.62:0/1779110332 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938007740 con 0x7fb9400d21b0 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.716+0000 7fb93effd640 1 -- 172.21.15.62:0/1779110332 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9380078e0 con 0x7fb9400d21b0 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.716+0000 7fb93effd640 1 -- 172.21.15.62:0/1779110332 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938007cb0 con 0x7fb9400d21b0 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 -- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d21b0 msgr2=0x7fb9400b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d21b0 0x7fb9400b7fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb938009d90 tx=0x7fb93802a660 comp rx=0 tx=0).stop 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 -- 172.21.15.62:0/1779110332 shutdown_connections 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d21b0 0x7fb9400b7fb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 --2- 172.21.15.62:0/1779110332 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 -- 172.21.15.62:0/1779110332 >> 172.21.15.62:0/1779110332 conn(0x7fb94009e510 msgr2=0x7fb9400d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 -- 172.21.15.62:0/1779110332 shutdown_connections 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.717+0000 7fb946522640 1 -- 172.21.15.62:0/1779110332 wait complete. 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.718+0000 7fb946522640 1 Processor -- start 2024-04-26T22:27:35.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.718+0000 7fb946522640 1 -- start start 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d0d30 0x7fb94015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb94015f850 con 0x7fb9400d21b0 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9400d3a50 con 0x7fb9400d0d30 2024-04-26T22:27:35.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb946522640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb9400d3f70 con 0x7fb94015f470 2024-04-26T22:27:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42178/0 (socket says 172.21.15.62:42178) 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 -- 172.21.15.62:0/2517593619 learned_addr learned my addr 172.21.15.62:0/2517593619 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93ffff640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d0d30 0x7fb94015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb944abf640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 -- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 msgr2=0x7fb9400ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400ba1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 -- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d0d30 msgr2=0x7fb94015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d0d30 0x7fb94015d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.719+0000 7fb93f7fe640 1 -- 172.21.15.62:0/2517593619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400ba7f0 con 0x7fb9400d21b0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb93f7fe640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c430 tx=0x7fb93400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9340074e0 con 0x7fb9400d21b0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9340047e0 con 0x7fb9400d21b0 2024-04-26T22:27:35.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934004bd0 con 0x7fb9400d21b0 2024-04-26T22:27:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9400baad0 con 0x7fb9400d21b0 2024-04-26T22:27:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.720+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9400cb6b0 con 0x7fb9400d21b0 2024-04-26T22:27:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.721+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94006a360 con 0x7fb9400d21b0 2024-04-26T22:27:35.724 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.721+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb934011070 con 0x7fb9400d21b0 2024-04-26T22:27:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.722+0000 7fb93cff9640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.722+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb93408b600 con 0x7fb9400d21b0 2024-04-26T22:27:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.722+0000 7fb93ffff640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:35.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.722+0000 7fb93ffff640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9401f8810 tx=0x7fb930009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:35.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.726+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb934059470 con 0x7fb9400d21b0 2024-04-26T22:27:35.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.912+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9400baca0 con 0x7fb9400d21b0 2024-04-26T22:27:35.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.912+0000 7fb93cff9640 1 -- 172.21.15.62:0/2517593619 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb93405d770 con 0x7fb9400d21b0 2024-04-26T22:27:35.913 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 msgr2=0x7fb92c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 0x7fb92c06edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9401f8810 tx=0x7fb930009190 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 msgr2=0x7fb9400b9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c430 tx=0x7fb93400c8f0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 shutdown_connections 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb92c06c8f0 0x7fb92c06edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb94015f470 0x7fb9400ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb9400d21b0 0x7fb9400b9c70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 --2- 172.21.15.62:0/2517593619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9400d0d30 0x7fb94015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:35.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.914+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 >> 172.21.15.62:0/2517593619 conn(0x7fb94009e510 msgr2=0x7fb9400d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.915+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 shutdown_connections 2024-04-26T22:27:35.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:35.915+0000 7fb946522640 1 -- 172.21.15.62:0/2517593619 wait complete. 2024-04-26T22:27:35.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:36.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.029+0000 7f89b782a640 1 Processor -- start 2024-04-26T22:27:36.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.029+0000 7f89b782a640 1 -- start start 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b782a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b006e700 0x7f89b00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b782a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89b00cd100 0x7f89b00ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b782a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b006ece0 con 0x7f89b015d460 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b782a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f89b00add80 con 0x7f89b00cd100 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b782a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f89b006efc0 con 0x7f89b006e700 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b5dc7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89b00cd100 0x7f89b00ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b5dc7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f89b00cd100 0x7f89b00ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42190/0 (socket says 172.21.15.62:42190) 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.030+0000 7f89b5dc7640 1 -- 172.21.15.62:0/3595877353 learned_addr learned my addr 172.21.15.62:0/3595877353 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b4dc5640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b006e700 0x7f89b00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b5dc7640 1 -- 172.21.15.62:0/3595877353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b006e700 msgr2=0x7f89b00ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:36.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b5dc7640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b006e700 0x7f89b00ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b5dc7640 1 -- 172.21.15.62:0/3595877353 >> v1:172.21.15.174:6789/0 conn(0x7f89b015d460 legacy=0x7f89b0073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:36.032 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b5dc7640 1 -- 172.21.15.62:0/3595877353 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b006f2a0 con 0x7f89b00cd100 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89b5dc7640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b00cd100 0x7f89b00ae210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009e50 tx=0x7f89a002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a649f083e77bf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.031+0000 7f89affff640 1 -- 172.21.15.62:0/3595877353 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0036070 con 0x7f89b00cd100 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89affff640 1 -- 172.21.15.62:0/3595877353 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a0031600 con 0x7f89b00cd100 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89affff640 1 -- 172.21.15.62:0/3595877353 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00319d0 con 0x7f89b00cd100 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 -- 172.21.15.62:0/3595877353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b00cd100 msgr2=0x7f89b00ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b00cd100 0x7f89b00ae210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009e50 tx=0x7f89a002a750 comp rx=0 tx=0).stop 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 -- 172.21.15.62:0/3595877353 shutdown_connections 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b00cd100 0x7f89b00ae210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 --2- 172.21.15.62:0/3595877353 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b006e700 0x7f89b00ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 -- 172.21.15.62:0/3595877353 >> 172.21.15.62:0/3595877353 conn(0x7f89b00d0d70 msgr2=0x7f89b00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 -- 172.21.15.62:0/3595877353 shutdown_connections 2024-04-26T22:27:36.034 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.032+0000 7f89b782a640 1 -- 172.21.15.62:0/3595877353 wait complete. 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.033+0000 7f89b782a640 1 Processor -- start 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.033+0000 7f89b782a640 1 -- start start 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b00cd100 0x7f89b015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b015d460 0x7f89b015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89b015d9c0 con 0x7f89b006e700 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89b015d840 con 0x7f89b015d460 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b782a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f89b00ae750 con 0x7f89b00cd100 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b55c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.036 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b55c6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42206/0 (socket says 172.21.15.62:42206) 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b55c6640 1 -- 172.21.15.62:0/3956836675 learned_addr learned my addr 172.21.15.62:0/3956836675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b4dc5640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b00cd100 0x7f89b015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.034+0000 7f89b5dc7640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b015d460 0x7f89b015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 -- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b00cd100 msgr2=0x7f89b015eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b00cd100 0x7f89b015eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 -- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b015d460 msgr2=0x7f89b015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b015d460 0x7f89b015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 -- 172.21.15.62:0/3956836675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b01f4180 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b55c6640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b3f0 tx=0x7f89a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8015020 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8011830 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b00aef80 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011c20 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.035+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b00af540 con 0x7f89b006e700 2024-04-26T22:27:36.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.036+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b006a360 con 0x7f89b006e700 2024-04-26T22:27:36.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.036+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89a8018020 con 0x7f89b006e700 2024-04-26T22:27:36.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.037+0000 7f89adffb640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.037+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89a808b5b0 con 0x7f89b006e700 2024-04-26T22:27:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.037+0000 7f89b4dc5640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.038+0000 7f89b4dc5640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 0x7f899c06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89a400aac0 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.041+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89a801f030 con 0x7f89b006e700 2024-04-26T22:27:36.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.225+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f89b00ce470 con 0x7f89b006e700 2024-04-26T22:27:36.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.225+0000 7f89adffb640 1 -- 172.21.15.62:0/3956836675 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f89b01f4180 con 0x7f89b006e700 2024-04-26T22:27:36.225 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:36.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 msgr2=0x7f899c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 0x7f899c06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f89a400aac0 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2024-04-26T22:27:36.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 msgr2=0x7f89b01f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b3f0 tx=0x7f89a800b8b0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 shutdown_connections 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f899c06c8f0 0x7f899c06edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b015d460 0x7f89b015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89b00cd100 0x7f89b015eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 --2- 172.21.15.62:0/3956836675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89b006e700 0x7f89b01f3c40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 >> 172.21.15.62:0/3956836675 conn(0x7f89b00d0d70 msgr2=0x7f89b015fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 shutdown_connections 2024-04-26T22:27:36.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.227+0000 7f89b782a640 1 -- 172.21.15.62:0/3956836675 wait complete. 2024-04-26T22:27:36.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:36.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.338+0000 7f205fe64640 1 Processor -- start 2024-04-26T22:27:36.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.338+0000 7f205fe64640 1 -- start start 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205fe64640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f205806fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205fe64640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2058160320 0x7f20580ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205fe64640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2058072e30 con 0x7f2058070130 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205fe64640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2058073110 con 0x7f2058160320 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205fe64640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f20580733f0 con 0x7f2058159a90 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205d3ff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2058070130 0x7f205815fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39764/0 (socket says 172.21.15.62:39764) 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205d3ff640 1 -- 172.21.15.62:0/4079437379 learned_addr learned my addr 172.21.15.62:0/4079437379 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.339+0000 7f205dc00640 1 --2- 172.21.15.62:0/4079437379 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f205806fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205e401640 1 --2- 172.21.15.62:0/4079437379 >> v2:172.21.15.174:3300/0 conn(0x7f2058160320 0x7f20580ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205cbfe640 1 -- 172.21.15.62:0/4079437379 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640506765 0 0) 0x7f2058072e30 con 0x7f2058070130 2024-04-26T22:27:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205cbfe640 1 -- 172.21.15.62:0/4079437379 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2044003520 con 0x7f2058070130 2024-04-26T22:27:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205dc00640 1 -- 172.21.15.62:0/4079437379 >> v2:172.21.15.174:3300/0 conn(0x7f2058160320 msgr2=0x7f20580ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205dc00640 1 --2- 172.21.15.62:0/4079437379 >> v2:172.21.15.174:3300/0 conn(0x7f2058160320 0x7f20580ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205dc00640 1 -- 172.21.15.62:0/4079437379 >> v1:172.21.15.62:6789/0 conn(0x7f2058070130 legacy=0x7f205815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.340+0000 7f205dc00640 1 -- 172.21.15.62:0/4079437379 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20580736d0 con 0x7f2058159a90 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205dc00640 1 --2- 172.21.15.62:0/4079437379 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f205806fbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d861212c435f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205cbfe640 1 -- 172.21.15.62:0/4079437379 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054037070 con 0x7f2058159a90 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205cbfe640 1 -- 172.21.15.62:0/4079437379 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205402adb0 con 0x7f2058159a90 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205cbfe640 1 -- 172.21.15.62:0/4079437379 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054032640 con 0x7f2058159a90 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 -- 172.21.15.62:0/4079437379 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 msgr2=0x7f205806fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 --2- 172.21.15.62:0/4079437379 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f205806fbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20540097a0 tx=0x7f205402a060 comp rx=0 tx=0).stop 2024-04-26T22:27:36.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 -- 172.21.15.62:0/4079437379 shutdown_connections 2024-04-26T22:27:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 --2- 172.21.15.62:0/4079437379 >> v2:172.21.15.174:3300/0 conn(0x7f2058160320 0x7f20580ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 --2- 172.21.15.62:0/4079437379 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f205806fbf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.341+0000 7f205fe64640 1 -- 172.21.15.62:0/4079437379 >> 172.21.15.62:0/4079437379 conn(0x7f20580d0d50 msgr2=0x7f20580d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.342+0000 7f205fe64640 1 -- 172.21.15.62:0/4079437379 shutdown_connections 2024-04-26T22:27:36.343 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.342+0000 7f205fe64640 1 -- 172.21.15.62:0/4079437379 wait complete. 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.343+0000 7f205fe64640 1 Processor -- start 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.343+0000 7f205fe64640 1 -- start start 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f20580b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 0x7f2058071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2058159ff0 con 0x7f2058070130 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2058159e70 con 0x7f2058160320 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f20580ceef0 con 0x7f2058159a90 2024-04-26T22:27:36.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42220/0 (socket says 172.21.15.62:42220) 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 -- 172.21.15.62:0/3961075727 learned_addr learned my addr 172.21.15.62:0/3961075727 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205d3ff640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f20580b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 -- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 msgr2=0x7f20580b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205e401640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 0x7f2058071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f20580b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 -- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 msgr2=0x7f2058071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 0x7f2058071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 -- 172.21.15.62:0/3961075727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20581f67b0 con 0x7f2058070130 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205d3ff640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f20580b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205e401640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 0x7f2058071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205dc00640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f205402a0a0 tx=0x7f20540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.346 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054041070 con 0x7f2058070130 2024-04-26T22:27:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205403c680 con 0x7f2058070130 2024-04-26T22:27:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2058149db0 con 0x7f2058070130 2024-04-26T22:27:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2054037040 con 0x7f2058070130 2024-04-26T22:27:36.347 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.344+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20581f5ce0 con 0x7f2058070130 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.345+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205806a360 con 0x7f2058070130 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.346+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2054044070 con 0x7f2058070130 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.346+0000 7f204e7fc640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 0x7f203c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.347+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f20540ac1f0 con 0x7f2058070130 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.347+0000 7f205d3ff640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 0x7f203c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.347+0000 7f205d3ff640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 0x7f203c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2058072b30 tx=0x7f2048009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.351+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f205407a5a0 con 0x7f2058070130 2024-04-26T22:27:36.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.538+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20580d0730 con 0x7f2058070130 2024-04-26T22:27:36.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.539+0000 7f204e7fc640 1 -- 172.21.15.62:0/3961075727 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f2054079ea0 con 0x7f2058070130 2024-04-26T22:27:36.539 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:36.539 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.540+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 msgr2=0x7f203c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.540+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 0x7f203c06edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2058072b30 tx=0x7f2048009040 comp rx=0 tx=0).stop 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.540+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 msgr2=0x7f2058149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.540+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f205402a0a0 tx=0x7f20540046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 shutdown_connections 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f203c06c8f0 0x7f203c06edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2058160320 0x7f2058071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2058159a90 0x7f20580b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 --2- 172.21.15.62:0/3961075727 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2058070130 0x7f2058149870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 >> 172.21.15.62:0/3961075727 conn(0x7f20580d0d50 msgr2=0x7f20580b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 shutdown_connections 2024-04-26T22:27:36.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.541+0000 7f205fe64640 1 -- 172.21.15.62:0/3961075727 wait complete. 2024-04-26T22:27:36.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:36.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:36.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.657+0000 7fc2884da640 1 Processor -- start 2024-04-26T22:27:36.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.657+0000 7fc2884da640 1 -- start start 2024-04-26T22:27:36.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc2884da640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc28015f470 0x7fc2800d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc2884da640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc2800d0d30 0x7fc2800d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc2884da640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2800d2ed0 con 0x7fc2800d1fe0 2024-04-26T22:27:36.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc2884da640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800d1190 con 0x7fc2800d0d30 2024-04-26T22:27:36.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc2884da640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc2800d31b0 con 0x7fc28015f470 2024-04-26T22:27:36.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc286a77640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fc2800d1fe0 0x7fc280162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:36790/0 (socket says 172.21.15.62:36790) 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc285a75640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc2800d0d30 0x7fc2800d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc286276640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc28015f470 0x7fc2800d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.658+0000 7fc286a77640 1 -- 172.21.15.62:0/1841354193 learned_addr learned my addr 172.21.15.62:0/1841354193 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285274640 1 -- 172.21.15.62:0/1841354193 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268579528 0 0) 0x7fc2800d2ed0 con 0x7fc2800d1fe0 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285274640 1 -- 172.21.15.62:0/1841354193 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc26c003520 con 0x7fc2800d1fe0 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285a75640 1 -- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc28015f470 msgr2=0x7fc2800d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285a75640 1 --2- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc28015f470 0x7fc2800d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285a75640 1 -- 172.21.15.62:0/1841354193 >> v1:172.21.15.174:6789/0 conn(0x7fc2800d1fe0 legacy=0x7fc280162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285a75640 1 -- 172.21.15.62:0/1841354193 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800d3490 con 0x7fc2800d0d30 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.659+0000 7fc285a75640 1 --2- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d0d30 0x7fc2800d1aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2700097d0 tx=0x7fc27002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ebb5d692321e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc285274640 1 -- 172.21.15.62:0/1841354193 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270037070 con 0x7fc2800d0d30 2024-04-26T22:27:36.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc285274640 1 -- 172.21.15.62:0/1841354193 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc270032440 con 0x7fc2800d0d30 2024-04-26T22:27:36.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc285274640 1 -- 172.21.15.62:0/1841354193 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2700327f0 con 0x7fc2800d0d30 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 -- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d0d30 msgr2=0x7fc2800d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 --2- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d0d30 0x7fc2800d1aa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc2700097d0 tx=0x7fc27002a1c0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 -- 172.21.15.62:0/1841354193 shutdown_connections 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 --2- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d0d30 0x7fc2800d1aa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 --2- 172.21.15.62:0/1841354193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc28015f470 0x7fc2800d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 -- 172.21.15.62:0/1841354193 >> 172.21.15.62:0/1841354193 conn(0x7fc28009e510 msgr2=0x7fc2800d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 -- 172.21.15.62:0/1841354193 shutdown_connections 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.660+0000 7fc2884da640 1 -- 172.21.15.62:0/1841354193 wait complete. 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.661+0000 7fc2884da640 1 Processor -- start 2024-04-26T22:27:36.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 -- start start 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2800d0d30 0x7fc28015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d1fe0 0x7fc2801f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc28015f850 con 0x7fc28015f470 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800d3a50 con 0x7fc2800d1fe0 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc2884da640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc2800d3f70 con 0x7fc2800d0d30 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc286a77640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc286a77640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42236/0 (socket says 172.21.15.62:42236) 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc286a77640 1 -- 172.21.15.62:0/1644392136 learned_addr learned my addr 172.21.15.62:0/1644392136 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.662+0000 7fc286276640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2800d0d30 0x7fc28015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.665 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc285a75640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d1fe0 0x7fc2801f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 -- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2800d0d30 msgr2=0x7fc28015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2800d0d30 0x7fc28015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 -- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d1fe0 msgr2=0x7fc2801f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d1fe0 0x7fc2801f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 -- 172.21.15.62:0/1644392136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28015c530 con 0x7fc28015f470 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc286a77640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc27800caa0 tx=0x7fc27800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278017070 con 0x7fc28015f470 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27800ac00 con 0x7fc28015f470 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2800b8840 con 0x7fc28015f470 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278012400 con 0x7fc28015f470 2024-04-26T22:27:36.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.663+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2800b90e0 con 0x7fc28015f470 2024-04-26T22:27:36.667 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.664+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28006a360 con 0x7fc28015f470 2024-04-26T22:27:36.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.664+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc27801a070 con 0x7fc28015f470 2024-04-26T22:27:36.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.665+0000 7fc276ffd640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.665+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc27808b780 con 0x7fc28015f470 2024-04-26T22:27:36.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.665+0000 7fc286276640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.666+0000 7fc286276640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 0x7fc26406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009aa0 tx=0x7fc27c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.670 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.669+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2780595f0 con 0x7fc28015f470 2024-04-26T22:27:36.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.854+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2800d5870 con 0x7fc28015f470 2024-04-26T22:27:36.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.855+0000 7fc276ffd640 1 -- 172.21.15.62:0/1644392136 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc27805d8f0 con 0x7fc28015f470 2024-04-26T22:27:36.855 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:36.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 msgr2=0x7fc26406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 0x7fc26406edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009aa0 tx=0x7fc27c009190 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 msgr2=0x7fc2800b98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc27800caa0 tx=0x7fc27800cf60 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 shutdown_connections 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc26406c8f0 0x7fc26406edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc28015f470 0x7fc2800b98e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2800d1fe0 0x7fc2801f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 --2- 172.21.15.62:0/1644392136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc2800d0d30 0x7fc28015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 >> 172.21.15.62:0/1644392136 conn(0x7fc28009e510 msgr2=0x7fc2800d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 shutdown_connections 2024-04-26T22:27:36.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.857+0000 7fc2884da640 1 -- 172.21.15.62:0/1644392136 wait complete. 2024-04-26T22:27:36.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.973+0000 7fabce48c640 1 Processor -- start 2024-04-26T22:27:36.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.973+0000 7fabce48c640 1 -- start start 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabce48c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabc80d5a80 0x7fabc80b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabce48c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabc80b4b20 0x7fabc81630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabce48c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80b2760 con 0x7fabc815f400 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabce48c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fabc80b43e0 con 0x7fabc80b4b20 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabce48c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fabc80b3ed0 con 0x7fabc80d5a80 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabc80b4b20 0x7fabc81630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fabc80b4b20 0x7fabc81630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42244/0 (socket says 172.21.15.62:42244) 2024-04-26T22:27:36.975 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 -- 172.21.15.62:0/3022318785 learned_addr learned my addr 172.21.15.62:0/3022318785 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabc77fe640 1 --2- 172.21.15.62:0/3022318785 >> v2:172.21.15.174:3300/0 conn(0x7fabc80d5a80 0x7fabc80b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 -- 172.21.15.62:0/3022318785 >> v2:172.21.15.174:3300/0 conn(0x7fabc80d5a80 msgr2=0x7fabc80b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 --2- 172.21.15.62:0/3022318785 >> v2:172.21.15.174:3300/0 conn(0x7fabc80d5a80 0x7fabc80b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 -- 172.21.15.62:0/3022318785 >> v1:172.21.15.174:6789/0 conn(0x7fabc815f400 legacy=0x7fabc80b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.974+0000 7fabcca29640 1 -- 172.21.15.62:0/3022318785 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80b2a00 con 0x7fabc80b4b20 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabcca29640 1 --2- 172.21.15.62:0/3022318785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80b4b20 0x7fabc81630b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabc808ea70 tx=0x7fabc002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac96d85594510770 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabc6ffd640 1 -- 172.21.15.62:0/3022318785 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0036070 con 0x7fabc80b4b20 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabc6ffd640 1 -- 172.21.15.62:0/3022318785 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc0031600 con 0x7fabc80b4b20 2024-04-26T22:27:36.976 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabc6ffd640 1 -- 172.21.15.62:0/3022318785 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc00319d0 con 0x7fabc80b4b20 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabce48c640 1 -- 172.21.15.62:0/3022318785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80b4b20 msgr2=0x7fabc81630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabce48c640 1 --2- 172.21.15.62:0/3022318785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80b4b20 0x7fabc81630b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fabc808ea70 tx=0x7fabc002a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.975+0000 7fabce48c640 1 -- 172.21.15.62:0/3022318785 shutdown_connections 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.976+0000 7fabce48c640 1 --2- 172.21.15.62:0/3022318785 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80b4b20 0x7fabc81630b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.976+0000 7fabce48c640 1 --2- 172.21.15.62:0/3022318785 >> v2:172.21.15.174:3300/0 conn(0x7fabc80d5a80 0x7fabc80b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.976+0000 7fabce48c640 1 -- 172.21.15.62:0/3022318785 >> 172.21.15.62:0/3022318785 conn(0x7fabc809e4f0 msgr2=0x7fabc80d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:36.979 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.976+0000 7fabce48c640 1 -- 172.21.15.62:0/3022318785 shutdown_connections 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.976+0000 7fabce48c640 1 -- 172.21.15.62:0/3022318785 wait complete. 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.977+0000 7fabce48c640 1 Processor -- start 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.977+0000 7fabce48c640 1 -- start start 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc80b4b20 0x7fabc8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 0x7fabc8160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80b32a0 con 0x7fabc80d5a80 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80b2fc0 con 0x7fabc80b4b20 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80b4f30 con 0x7fabc815f400 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42258/0 (socket says 172.21.15.62:42258) 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 -- 172.21.15.62:0/4169983921 learned_addr learned my addr 172.21.15.62:0/4169983921 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:36.980 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc7fff640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc80b4b20 0x7fabc8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabcca29640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 0x7fabc8160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 -- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 msgr2=0x7fabc8160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 0x7fabc8160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 -- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc80b4b20 msgr2=0x7fabc8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc80b4b20 0x7fabc8159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 -- 172.21.15.62:0/4169983921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80ca760 con 0x7fabc80d5a80 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabcca29640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 0x7fabc8160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc77fe640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d5f0 tx=0x7fabb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80125f0 con 0x7fabc80d5a80 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb8019070 con 0x7fabc80d5a80 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc80cb120 con 0x7fabc80d5a80 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.978+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80129d0 con 0x7fabc80d5a80 2024-04-26T22:27:36.981 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.979+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc80c9740 con 0x7fabc80d5a80 2024-04-26T22:27:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.979+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc806a360 con 0x7fabc80d5a80 2024-04-26T22:27:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.980+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabb8012b70 con 0x7fabc80d5a80 2024-04-26T22:27:36.982 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.980+0000 7fabc4ff9640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 0x7fabb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.980+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fabb808bf50 con 0x7fabc80d5a80 2024-04-26T22:27:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.981+0000 7fabc7fff640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 0x7fabb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:36.983 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.981+0000 7fabc7fff640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 0x7fabb406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000f60 tx=0x7fabbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:36.984 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:36.985+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabc80ca760 con 0x7fabc80d5a80 2024-04-26T22:27:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.167+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fabc80d5270 con 0x7fabc80d5a80 2024-04-26T22:27:37.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.168+0000 7fabc4ff9640 1 -- 172.21.15.62:0/4169983921 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fabb805a380 con 0x7fabc80d5a80 2024-04-26T22:27:37.168 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 msgr2=0x7fabb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 0x7fabb406edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fabbc000f60 tx=0x7fabbc009040 comp rx=0 tx=0).stop 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 msgr2=0x7fabc80ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d5f0 tx=0x7fabb800dab0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 shutdown_connections 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabb406c8f0 0x7fabb406edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabc815f400 0x7fabc8160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.170 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabc80d5a80 0x7fabc80ca220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 --2- 172.21.15.62:0/4169983921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc80b4b20 0x7fabc8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 >> 172.21.15.62:0/4169983921 conn(0x7fabc809e4f0 msgr2=0x7fabc80d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 shutdown_connections 2024-04-26T22:27:37.171 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.170+0000 7fabce48c640 1 -- 172.21.15.62:0/4169983921 wait complete. 2024-04-26T22:27:37.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.284+0000 7f65fe8fc640 1 Processor -- start 2024-04-26T22:27:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.284+0000 7f65fe8fc640 1 -- start start 2024-04-26T22:27:37.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fe8fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fe8fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65f80d2150 0x7f65f80d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fe8fc640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f8161510 con 0x7f65f8160800 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fe8fc640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f65f80d2590 con 0x7f65f80d2150 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fe8fc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f65f80d0930 con 0x7f65f81610c0 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fce99640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65f80d2150 0x7f65f80d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fce99640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f65f80d2150 0x7f65f80d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42264/0 (socket says 172.21.15.62:42264) 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65fce99640 1 -- 172.21.15.62:0/2625563642 learned_addr learned my addr 172.21.15.62:0/2625563642 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.285+0000 7f65f77fe640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.286+0000 7f65fce99640 1 -- 172.21.15.62:0/2625563642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 msgr2=0x7f65f80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.286+0000 7f65fce99640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.286+0000 7f65fce99640 1 -- 172.21.15.62:0/2625563642 >> v1:172.21.15.174:6789/0 conn(0x7f65f8160800 legacy=0x7f65f80d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.286+0000 7f65fce99640 1 -- 172.21.15.62:0/2625563642 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f80d0c00 con 0x7f65f80d2150 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.286+0000 7f65fce99640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f80d2150 0x7f65f80d3610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009e50 tx=0x7f65f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae20a031876a629c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65f6ffd640 1 -- 172.21.15.62:0/2625563642 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0036070 con 0x7f65f80d2150 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65f6ffd640 1 -- 172.21.15.62:0/2625563642 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f0031600 con 0x7f65f80d2150 2024-04-26T22:27:37.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65f6ffd640 1 -- 172.21.15.62:0/2625563642 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f00319d0 con 0x7f65f80d2150 2024-04-26T22:27:37.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2625563642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f80d2150 msgr2=0x7f65f80d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f80d2150 0x7f65f80d3610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009e50 tx=0x7f65f002a710 comp rx=0 tx=0).stop 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2625563642 shutdown_connections 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f80d2150 0x7f65f80d3610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2625563642 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2625563642 >> 172.21.15.62:0/2625563642 conn(0x7f65f809e550 msgr2=0x7f65f80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2625563642 shutdown_connections 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.287+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2625563642 wait complete. 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.288+0000 7f65fe8fc640 1 Processor -- start 2024-04-26T22:27:37.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.288+0000 7f65fe8fc640 1 -- start start 2024-04-26T22:27:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65f80d2150 0x7f65f81f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.291 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80d2e10 con 0x7f65f8160800 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80d11c0 con 0x7f65f80d2150 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fe8fc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f65f80d30f0 con 0x7f65f81610c0 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65f77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65f77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42272/0 (socket says 172.21.15.62:42272) 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65f77fe640 1 -- 172.21.15.62:0/2937194178 learned_addr learned my addr 172.21.15.62:0/2937194178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65f7fff640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65f80d2150 0x7f65f81f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.289+0000 7f65fce99640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 -- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 msgr2=0x7f65f80b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 -- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65f80d2150 msgr2=0x7f65f81f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65f80d2150 0x7f65f81f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 -- 172.21.15.62:0/2937194178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f81f58b0 con 0x7f65f8160800 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f77fe640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00e870 tx=0x7f65ec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x7f65f8160800 2024-04-26T22:27:37.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec00c940 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f80b5340 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec00cd30 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.290+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f80b5900 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.291+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f806a360 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.291+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65ec01a070 con 0x7f65f8160800 2024-04-26T22:27:37.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.292+0000 7f65f4ff9640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.292+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65ec08b8e0 con 0x7f65f8160800 2024-04-26T22:27:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.292+0000 7f65f7fff640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.293+0000 7f65f7fff640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 0x7f65e406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001780 tx=0x7f65e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.296+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65ec059750 con 0x7f65f8160800 2024-04-26T22:27:37.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.482+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65f80d4530 con 0x7f65f8160800 2024-04-26T22:27:37.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.482+0000 7f65f4ff9640 1 -- 172.21.15.62:0/2937194178 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f65ec05da50 con 0x7f65f8160800 2024-04-26T22:27:37.482 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 msgr2=0x7f65e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 0x7f65e406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001780 tx=0x7f65e8009040 comp rx=0 tx=0).stop 2024-04-26T22:27:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 msgr2=0x7f65f81f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00e870 tx=0x7f65ec00ed30 comp rx=0 tx=0).stop 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 shutdown_connections 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f65e406c920 0x7f65e406ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65f81610c0 0x7f65f80b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65f8160800 0x7f65f81f5370 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 --2- 172.21.15.62:0/2937194178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65f80d2150 0x7f65f81f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 >> 172.21.15.62:0/2937194178 conn(0x7f65f809e550 msgr2=0x7f65f8157820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 shutdown_connections 2024-04-26T22:27:37.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.484+0000 7f65fe8fc640 1 -- 172.21.15.62:0/2937194178 wait complete. 2024-04-26T22:27:37.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:37.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.596+0000 7f9dd28d1640 1 Processor -- start 2024-04-26T22:27:37.597 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.596+0000 7f9dd28d1640 1 -- start start 2024-04-26T22:27:37.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd28d1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9dcc163560 0x7f9dcc0d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd28d1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc1639e0 0x7f9dcc0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd28d1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dcc0cfea0 con 0x7f9dcc15f470 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd28d1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f9dcc0d0a70 con 0x7f9dcc163560 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd28d1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc0d0140 con 0x7f9dcc1639e0 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dcb7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9dcc163560 0x7f9dcc0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dcb7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f9dcc163560 0x7f9dcc0d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42276/0 (socket says 172.21.15.62:42276) 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dcb7fe640 1 -- 172.21.15.62:0/892105973 learned_addr learned my addr 172.21.15.62:0/892105973 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.597+0000 7f9dd0e6e640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc1639e0 0x7f9dcc0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcb7fe640 1 -- 172.21.15.62:0/892105973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc1639e0 msgr2=0x7f9dcc0d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:37.599 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcb7fe640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc1639e0 0x7f9dcc0d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcb7fe640 1 -- 172.21.15.62:0/892105973 >> v1:172.21.15.174:6789/0 conn(0x7f9dcc15f470 legacy=0x7f9dcc162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcb7fe640 1 -- 172.21.15.62:0/892105973 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc0d0420 con 0x7f9dcc163560 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcb7fe640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc163560 0x7f9dcc0d2d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009870 tx=0x7f9dc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2540ebd0ff52e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcaffd640 1 -- 172.21.15.62:0/892105973 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0037070 con 0x7f9dcc163560 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcaffd640 1 -- 172.21.15.62:0/892105973 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc0032440 con 0x7f9dcc163560 2024-04-26T22:27:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.598+0000 7f9dcaffd640 1 -- 172.21.15.62:0/892105973 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0032810 con 0x7f9dcc163560 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 -- 172.21.15.62:0/892105973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc163560 msgr2=0x7f9dcc0d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc163560 0x7f9dcc0d2d30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009870 tx=0x7f9dc002a270 comp rx=0 tx=0).stop 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 -- 172.21.15.62:0/892105973 shutdown_connections 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc1639e0 0x7f9dcc0d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/892105973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc163560 0x7f9dcc0d2d30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 -- 172.21.15.62:0/892105973 >> 172.21.15.62:0/892105973 conn(0x7f9dcc09e4f0 msgr2=0x7f9dcc0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 -- 172.21.15.62:0/892105973 shutdown_connections 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.599+0000 7f9dd28d1640 1 -- 172.21.15.62:0/892105973 wait complete. 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.600+0000 7f9dd28d1640 1 Processor -- start 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.600+0000 7f9dd28d1640 1 -- start start 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc163560 0x7f9dcc1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dcc1639e0 0x7f9dcc1f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc0d5e10 con 0x7f9dcc15f470 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc0d5c90 con 0x7f9dcc1639e0 2024-04-26T22:27:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd28d1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc0d0f00 con 0x7f9dcc163560 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dcbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42288/0 (socket says 172.21.15.62:42288) 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dcbfff640 1 -- 172.21.15.62:0/821953675 learned_addr learned my addr 172.21.15.62:0/821953675 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dcb7fe640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc163560 0x7f9dcc1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.601+0000 7f9dd0e6e640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dcc1639e0 0x7f9dcc1f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 -- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc163560 msgr2=0x7f9dcc1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc163560 0x7f9dcc1f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 -- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dcc1639e0 msgr2=0x7f9dcc1f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dcc1639e0 0x7f9dcc1f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 -- 172.21.15.62:0/821953675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc1f5a50 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dcbfff640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c400 tx=0x7f9dbc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0052d0 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc0054b0 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc1f5d30 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc006a40 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.602+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc160d00 con 0x7f9dcc15f470 2024-04-26T22:27:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.603+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dcc1f6250 con 0x7f9dcc15f470 2024-04-26T22:27:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.604+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dbc006be0 con 0x7f9dcc15f470 2024-04-26T22:27:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.604+0000 7f9dc8ff9640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 0x7f9db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.604+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9dbc08afc0 con 0x7f9dcc15f470 2024-04-26T22:27:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.604+0000 7f9dcb7fe640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 0x7f9db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.605+0000 7f9dcb7fe640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 0x7f9db806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0034000 tx=0x7f9dc002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.609+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9dcc1f6250 con 0x7f9dcc15f470 2024-04-26T22:27:37.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.797+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dcc06a360 con 0x7f9dcc15f470 2024-04-26T22:27:37.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.798+0000 7f9dc8ff9640 1 -- 172.21.15.62:0/821953675 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f9dbc058da0 con 0x7f9dcc15f470 2024-04-26T22:27:37.798 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:37.798 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 msgr2=0x7f9db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 0x7f9db806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0034000 tx=0x7f9dc002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 msgr2=0x7f9dcc153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c400 tx=0x7f9dbc00c8c0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 shutdown_connections 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9db806c8f0 0x7f9db806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dcc1639e0 0x7f9dcc1f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9dcc163560 0x7f9dcc1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 --2- 172.21.15.62:0/821953675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9dcc15f470 0x7f9dcc153890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 >> 172.21.15.62:0/821953675 conn(0x7f9dcc09e4f0 msgr2=0x7f9dcc0b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 shutdown_connections 2024-04-26T22:27:37.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.800+0000 7f9dd28d1640 1 -- 172.21.15.62:0/821953675 wait complete. 2024-04-26T22:27:37.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:37.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:37.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.918+0000 7f8918e47640 1 Processor -- start 2024-04-26T22:27:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.918+0000 7f8918e47640 1 -- start start 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8918e47640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8914160800 0x7f89141610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8918e47640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89140d2320 0x7f89140d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8918e47640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89140d0930 con 0x7f89140d5b60 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8918e47640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f89140d0c00 con 0x7f8914160800 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8918e47640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f89140d0ee0 con 0x7f89140d2320 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f891259c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8914160800 0x7f89141610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f891259c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8914160800 0x7f89141610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42302/0 (socket says 172.21.15.62:42302) 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f891259c640 1 -- 172.21.15.62:0/3698396133 learned_addr learned my addr 172.21.15.62:0/3698396133 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.919+0000 7f8912d9d640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89140d2320 0x7f89140d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891259c640 1 -- 172.21.15.62:0/3698396133 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89140d2320 msgr2=0x7f89140d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891259c640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89140d2320 0x7f89140d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891259c640 1 -- 172.21.15.62:0/3698396133 >> v1:172.21.15.174:6789/0 conn(0x7f89140d5b60 legacy=0x7f89140d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891259c640 1 -- 172.21.15.62:0/3698396133 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89140d11c0 con 0x7f8914160800 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891259c640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8914160800 0x7f89141610c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e50 tx=0x7f890402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a442ffedc51b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891159a640 1 -- 172.21.15.62:0/3698396133 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904036070 con 0x7f8914160800 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891159a640 1 -- 172.21.15.62:0/3698396133 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904031600 con 0x7f8914160800 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.920+0000 7f891159a640 1 -- 172.21.15.62:0/3698396133 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89040319d0 con 0x7f8914160800 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 -- 172.21.15.62:0/3698396133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8914160800 msgr2=0x7f89141610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8914160800 0x7f89141610c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e50 tx=0x7f890402a710 comp rx=0 tx=0).stop 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 -- 172.21.15.62:0/3698396133 shutdown_connections 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f89140d2320 0x7f89140d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 --2- 172.21.15.62:0/3698396133 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8914160800 0x7f89141610c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 -- 172.21.15.62:0/3698396133 >> 172.21.15.62:0/3698396133 conn(0x7f891409e550 msgr2=0x7f89140d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 -- 172.21.15.62:0/3698396133 shutdown_connections 2024-04-26T22:27:37.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.921+0000 7f8918e47640 1 -- 172.21.15.62:0/3698396133 wait complete. 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.922+0000 7f8918e47640 1 Processor -- start 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 -- start start 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89140d5b60 0x7f89140b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8914160800 0x7f89140b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f89140b6300 con 0x7f89140d2320 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89140d30f0 con 0x7f89140d5b60 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8918e47640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8914160be0 con 0x7f8914160800 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f891259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f891259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42316/0 (socket says 172.21.15.62:42316) 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f891259c640 1 -- 172.21.15.62:0/59055719 learned_addr learned my addr 172.21.15.62:0/59055719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8911d9b640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89140d5b60 0x7f89140b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.923+0000 7f8912d9d640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8914160800 0x7f89140b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 -- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8914160800 msgr2=0x7f89140b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8914160800 0x7f89140b9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 -- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89140d5b60 msgr2=0x7f89140b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89140d5b60 0x7f89140b9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 -- 172.21.15.62:0/59055719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89140b80b0 con 0x7f89140d2320 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f891259c640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8904034040 tx=0x7f8904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904036050 con 0x7f89140d2320 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904042070 con 0x7f89140d2320 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89140b8390 con 0x7f89140d2320 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904009510 con 0x7f89140d2320 2024-04-26T22:27:37.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.924+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89140b8950 con 0x7f89140d2320 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.925+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891406a360 con 0x7f89140d2320 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.926+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8904007c50 con 0x7f89140d2320 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.926+0000 7f8902ffd640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 0x7f88f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.926+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f89040ac8c0 con 0x7f89140d2320 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.926+0000 7f8911d9b640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 0x7f88f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:37.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.927+0000 7f8911d9b640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 0x7f88f406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89140cbb30 tx=0x7f8908009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:37.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:37.932+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f890407a6a0 con 0x7f89140d2320 2024-04-26T22:27:38.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.115+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89140d2ba0 con 0x7f89140d2320 2024-04-26T22:27:38.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.115+0000 7f8902ffd640 1 -- 172.21.15.62:0/59055719 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f890407e9a0 con 0x7f89140d2320 2024-04-26T22:27:38.116 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 msgr2=0x7f88f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 0x7f88f406edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f89140cbb30 tx=0x7f8908009040 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 msgr2=0x7f89140cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8904034040 tx=0x7f8904004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 shutdown_connections 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f88f406c8f0 0x7f88f406edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8914160800 0x7f89140b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89140d5b60 0x7f89140b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 --2- 172.21.15.62:0/59055719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f89140d2320 0x7f89140cadb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 >> 172.21.15.62:0/59055719 conn(0x7f891409e550 msgr2=0x7f8914161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 shutdown_connections 2024-04-26T22:27:38.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.117+0000 7f8918e47640 1 -- 172.21.15.62:0/59055719 wait complete. 2024-04-26T22:27:38.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.232+0000 7f3f344cf640 1 Processor -- start 2024-04-26T22:27:38.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.232+0000 7f3f344cf640 1 -- start start 2024-04-26T22:27:38.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f344cf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f2c0d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.236 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f344cf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f2c0d2ed0 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f344cf640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f2c0d31b0 con 0x7f3f2c15f470 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f344cf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d3490 con 0x7f3f2c0d0f00 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f32a6c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3f2c0d1fe0 0x7f3f2c0b7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39780/0 (socket says 172.21.15.62:39780) 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f32a6c640 1 -- 172.21.15.62:0/4237469404 learned_addr learned my addr 172.21.15.62:0/4237469404 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f31a6a640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f2c0d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251016681 0 0) 0x7f3f2c0d2ed0 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f18003520 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322829266 0 0) 0x7f3f2c0d31b0 con 0x7f3f2c15f470 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.233+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f2c0d2ed0 con 0x7f3f2c15f470 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1799406217 0 0) 0x7f3f18003520 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 msgr2=0x7f3f2c0d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:38.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f2c0d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 >> v1:172.21.15.174:6789/0 conn(0x7f3f2c15f470 legacy=0x7f3f2c0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c0d3770 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f3f240019f0 con 0x7f3f2c0d1fe0 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 >> v1:172.21.15.62:6789/0 conn(0x7f3f2c0d1fe0 legacy=0x7f3f2c0b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f18003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f18004800 0x7f3f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d3490 con 0x7f3f18003b70 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1800ba70 con 0x7f3f18004800 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1800bc70 con 0x7f3f2c0d0f00 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f18003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f31a6a640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f18004800 0x7f3f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f32a6c640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 -- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 msgr2=0x7f3f180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0d0f00 0x7f3f180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 -- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f18004800 msgr2=0x7f3f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f18004800 0x7f3f18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.234+0000 7f3f3226b640 1 -- 172.21.15.62:0/4237469404 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f18003520 con 0x7f3f18003b70 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f3226b640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f18003f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3f280018d0 tx=0x7f3f18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3479f4f793c8091 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f280014f0 con 0x7f3f18003b70 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f280052d0 con 0x7f3f18003b70 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f31269640 1 -- 172.21.15.62:0/4237469404 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2802aca0 con 0x7f3f18003b70 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f344cf640 1 -- 172.21.15.62:0/4237469404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 msgr2=0x7f3f18003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.235+0000 7f3f344cf640 1 --2- 172.21.15.62:0/4237469404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f18003f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3f280018d0 tx=0x7f3f18005830 comp rx=0 tx=0).stop 2024-04-26T22:27:38.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.236+0000 7f3f32a6c640 1 -- 172.21.15.62:0/4237469404 reap_dead start 2024-04-26T22:27:38.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.236+0000 7f3f344cf640 1 -- 172.21.15.62:0/4237469404 shutdown_connections 2024-04-26T22:27:38.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.236+0000 7f3f344cf640 1 -- 172.21.15.62:0/4237469404 >> 172.21.15.62:0/4237469404 conn(0x7f3f2c09e510 msgr2=0x7f3f2c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.236+0000 7f3f344cf640 1 -- 172.21.15.62:0/4237469404 shutdown_connections 2024-04-26T22:27:38.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.236+0000 7f3f344cf640 1 -- 172.21.15.62:0/4237469404 wait complete. 2024-04-26T22:27:38.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.237+0000 7f3f344cf640 1 Processor -- start 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.237+0000 7f3f344cf640 1 -- start start 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0ccdd0 0x7f3f2c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f2c0cadd0 0x7f3f2c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c0d3f70 con 0x7f3f18003b70 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c15f850 con 0x7f3f2c0cadd0 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c156e00 con 0x7f3f2c0ccdd0 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42336/0 (socket says 172.21.15.62:42336) 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 -- 172.21.15.62:0/573961967 learned_addr learned my addr 172.21.15.62:0/573961967 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 -- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0ccdd0 msgr2=0x7f3f2c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0ccdd0 0x7f3f2c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f32a6c640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f2c0cadd0 0x7f3f2c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 -- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f2c0cadd0 msgr2=0x7f3f2c0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f2c0cadd0 0x7f3f2c0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 -- 172.21.15.62:0/573961967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c0cd8e0 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f3226b640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005040 tx=0x7f3f280012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28037040 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28006e50 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c0cdbc0 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f280014f0 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.238+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c0ce180 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.239+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c06a360 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.240+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f28044020 con 0x7f3f18003b70 2024-04-26T22:27:38.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.240+0000 7f3f22ffd640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 0x7f3f1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.240+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f280ab960 con 0x7f3f18003b70 2024-04-26T22:27:38.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.241+0000 7f3f31a6a640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 0x7f3f1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.241+0000 7f3f31a6a640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 0x7f3f1006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c1f85b0 tx=0x7f3f1c009320 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.245 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.245+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3f28079eb0 con 0x7f3f18003b70 2024-04-26T22:27:38.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.455+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f2c0d5870 con 0x7f3f18003b70 2024-04-26T22:27:38.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.456+0000 7f3f22ffd640 1 -- 172.21.15.62:0/573961967 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3f280797b0 con 0x7f3f18003b70 2024-04-26T22:27:38.456 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 msgr2=0x7f3f1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 0x7f3f1006ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c1f85b0 tx=0x7f3f1c009320 comp rx=0 tx=0).stop 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 msgr2=0x7f3f2c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005040 tx=0x7f3f280012d0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 shutdown_connections 2024-04-26T22:27:38.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3f1006c920 0x7f3f1006ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f2c0cadd0 0x7f3f2c0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f2c0ccdd0 0x7f3f2c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 --2- 172.21.15.62:0/573961967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3f18003b70 0x7f3f2c1568c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 >> 172.21.15.62:0/573961967 conn(0x7f3f2c09e510 msgr2=0x7f3f2c0b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 shutdown_connections 2024-04-26T22:27:38.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.458+0000 7f3f344cf640 1 -- 172.21.15.62:0/573961967 wait complete. 2024-04-26T22:27:38.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:38.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.572+0000 7f51ced0a640 1 Processor -- start 2024-04-26T22:27:38.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.572+0000 7f51ced0a640 1 -- start start 2024-04-26T22:27:38.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.573+0000 7f51ced0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c80cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.573+0000 7f51ced0a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f51c81603a0 0x7f51c8160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.573+0000 7f51ced0a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c806f690 con 0x7f51c80cbd60 2024-04-26T22:27:38.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.573+0000 7f51ced0a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f51c806f970 con 0x7f51c81603a0 2024-04-26T22:27:38.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.573+0000 7f51ced0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f51c80735a0 con 0x7f51c815ffc0 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51c7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f51c81603a0 0x7f51c8160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51cd2a7640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f51c80cbd60 0x7f51c80cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:36830/0 (socket says 172.21.15.62:36830) 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51ccaa6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c80cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51cd2a7640 1 -- 172.21.15.62:0/236888061 learned_addr learned my addr 172.21.15.62:0/236888061 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51ccaa6640 1 -- 172.21.15.62:0/236888061 >> v2:172.21.15.174:3300/0 conn(0x7f51c81603a0 msgr2=0x7f51c8160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:38.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51c77fe640 1 -- 172.21.15.62:0/236888061 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472873580 0 0) 0x7f51c806f690 con 0x7f51c80cbd60 2024-04-26T22:27:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51ccaa6640 1 --2- 172.21.15.62:0/236888061 >> v2:172.21.15.174:3300/0 conn(0x7f51c81603a0 0x7f51c8160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51ccaa6640 1 -- 172.21.15.62:0/236888061 >> v1:172.21.15.174:6789/0 conn(0x7f51c80cbd60 legacy=0x7f51c80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.574+0000 7f51ccaa6640 1 -- 172.21.15.62:0/236888061 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c8073880 con 0x7f51c815ffc0 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51ccaa6640 1 --2- 172.21.15.62:0/236888061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c80cb7f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d6994b087d8316 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51c77fe640 1 -- 172.21.15.62:0/236888061 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0037070 con 0x7f51c815ffc0 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51c77fe640 1 -- 172.21.15.62:0/236888061 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b0032440 con 0x7f51c815ffc0 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51c77fe640 1 -- 172.21.15.62:0/236888061 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0032810 con 0x7f51c815ffc0 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51ced0a640 1 -- 172.21.15.62:0/236888061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 msgr2=0x7f51c80cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.575+0000 7f51ced0a640 1 --2- 172.21.15.62:0/236888061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c80cb7f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a180 comp rx=0 tx=0).stop 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 -- 172.21.15.62:0/236888061 shutdown_connections 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 --2- 172.21.15.62:0/236888061 >> v2:172.21.15.174:3300/0 conn(0x7f51c81603a0 0x7f51c8160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 --2- 172.21.15.62:0/236888061 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c80cb7f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 -- 172.21.15.62:0/236888061 >> 172.21.15.62:0/236888061 conn(0x7f51c80d0d70 msgr2=0x7f51c80d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 -- 172.21.15.62:0/236888061 shutdown_connections 2024-04-26T22:27:38.576 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.576+0000 7f51ced0a640 1 -- 172.21.15.62:0/236888061 wait complete. 2024-04-26T22:27:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.577+0000 7f51ced0a640 1 Processor -- start 2024-04-26T22:27:38.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.577+0000 7f51ced0a640 1 -- start start 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c81f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c81603a0 0x7f51c806e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8160d60 con 0x7f51c80cbd60 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51c806fc50 con 0x7f51c81603a0 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ced0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f51c8160ee0 con 0x7f51c815ffc0 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42350/0 (socket says 172.21.15.62:42350) 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 -- 172.21.15.62:0/262915903 learned_addr learned my addr 172.21.15.62:0/262915903 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51cd2a7640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c81603a0 0x7f51c806e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51c7fff640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c81f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 -- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 msgr2=0x7f51c81f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c81f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 -- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c81603a0 msgr2=0x7f51c806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c81603a0 0x7f51c806e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 -- 172.21.15.62:0/262915903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c80c9360 con 0x7f51c80cbd60 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51ccaa6640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51b0035040 tx=0x7f51b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0037050 con 0x7f51c80cbd60 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.578+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b0038040 con 0x7f51c80cbd60 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.579+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c80c96a0 con 0x7f51c80cbd60 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.579+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0004490 con 0x7f51c80cbd60 2024-04-26T22:27:38.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.579+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c80c9ee0 con 0x7f51c80cbd60 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.580+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c806a360 con 0x7f51c80cbd60 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.580+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51b003c6d0 con 0x7f51c80cbd60 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.580+0000 7f51c57fa640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 0x7f51a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.580+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51b00ac600 con 0x7f51c80cbd60 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.581+0000 7f51c7fff640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 0x7f51a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.581+0000 7f51c7fff640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 0x7f51a806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f51c81549a0 tx=0x7f51b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.585+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51c80c96a0 con 0x7f51c80cbd60 2024-04-26T22:27:38.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.770+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51c80b6cf0 con 0x7f51c80cbd60 2024-04-26T22:27:38.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.770+0000 7f51c57fa640 1 -- 172.21.15.62:0/262915903 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f51b007aa40 con 0x7f51c80cbd60 2024-04-26T22:27:38.771 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 msgr2=0x7f51a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 0x7f51a806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f51c81549a0 tx=0x7f51b8008040 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 msgr2=0x7f51c80ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51b0035040 tx=0x7f51b00046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 shutdown_connections 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f51a806c8f0 0x7f51a806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c81603a0 0x7f51c806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f51c815ffc0 0x7f51c81f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 --2- 172.21.15.62:0/262915903 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51c80cbd60 0x7f51c80ca400 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 >> 172.21.15.62:0/262915903 conn(0x7f51c80d0d70 msgr2=0x7f51c80cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 shutdown_connections 2024-04-26T22:27:38.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.772+0000 7f51ced0a640 1 -- 172.21.15.62:0/262915903 wait complete. 2024-04-26T22:27:38.849 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.849 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-04-26T22:27:38.849 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2024-04-26T22:27:38.850 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2024-04-26T22:27:38.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:38.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 Processor -- start 2024-04-26T22:27:38.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 -- start start 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f37940b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37940d3de0 con 0x7f379415f760 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37940d40c0 con 0x7f379415f380 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.952+0000 7f3799a46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f37940d43a0 con 0x7f37940d22e0 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37937fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f379415f760 0x7f37940b8fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39788/0 (socket says 172.21.15.62:39788) 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37937fe640 1 -- 172.21.15.62:0/3917953274 learned_addr learned my addr 172.21.15.62:0/3917953274 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:38.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37927fc640 1 --2- 172.21.15.62:0/3917953274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f37940b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596010193 0 0) 0x7f37940d3de0 con 0x7f379415f760 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f377c003520 con 0x7f379415f760 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931252895 0 0) 0x7f37940d40c0 con 0x7f379415f380 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37940d3de0 con 0x7f379415f380 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1645148019 0 0) 0x7f377c003520 con 0x7f379415f760 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37927fc640 1 -- 172.21.15.62:0/3917953274 >> v1:172.21.15.174:6789/0 conn(0x7f379415f380 legacy=0x7f37940d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37927fc640 1 -- 172.21.15.62:0/3917953274 >> v1:172.21.15.62:6789/0 conn(0x7f379415f760 legacy=0x7f37940b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.953+0000 7f37927fc640 1 -- 172.21.15.62:0/3917953274 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37940b8280 con 0x7f37940d22e0 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.954+0000 7f37927fc640 1 --2- 172.21.15.62:0/3917953274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f37940b8a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37880097d0 tx=0x7f378802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69458c5e04f06166 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.954+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788037070 con 0x7f37940d22e0 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.954+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378802acc0 con 0x7f37940d22e0 2024-04-26T22:27:38.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.954+0000 7f3791ffb640 1 -- 172.21.15.62:0/3917953274 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c0019f0 con 0x7f37940d22e0 2024-04-26T22:27:38.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 -- 172.21.15.62:0/3917953274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 msgr2=0x7f37940b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 --2- 172.21.15.62:0/3917953274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f37940b8a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37880097d0 tx=0x7f378802a230 comp rx=0 tx=0).stop 2024-04-26T22:27:38.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 -- 172.21.15.62:0/3917953274 shutdown_connections 2024-04-26T22:27:38.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 --2- 172.21.15.62:0/3917953274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f37940b8a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 -- 172.21.15.62:0/3917953274 >> 172.21.15.62:0/3917953274 conn(0x7f379409e560 msgr2=0x7f37940d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:38.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 -- 172.21.15.62:0/3917953274 shutdown_connections 2024-04-26T22:27:38.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.955+0000 7f3799a46640 1 -- 172.21.15.62:0/3917953274 wait complete. 2024-04-26T22:27:38.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.956+0000 7f3799a46640 1 Processor -- start 2024-04-26T22:27:38.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.956+0000 7f3799a46640 1 -- start start 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f3794157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 0x7f3794155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f37940d1280 con 0x7f379415f380 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37940d1100 con 0x7f379415f760 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3799a46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f37940d5b50 con 0x7f37940d22e0 2024-04-26T22:27:38.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42358/0 (socket says 172.21.15.62:42358) 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 -- 172.21.15.62:0/1361927167 learned_addr learned my addr 172.21.15.62:0/1361927167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 -- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 msgr2=0x7f3794157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3792ffd640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f3794157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37937fe640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 0x7f3794155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f3794157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 -- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 msgr2=0x7f3794155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 0x7f3794155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 -- 172.21.15.62:0/1361927167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37941f93e0 con 0x7f379415f380 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37937fe640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 0x7f3794155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f3792ffd640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f3794157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.957+0000 7f37927fc640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37880097d0 tx=0x7f37880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.958+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788041020 con 0x7f379415f380 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.958+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378803d640 con 0x7f379415f380 2024-04-26T22:27:38.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.958+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37941f8620 con 0x7f379415f380 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.958+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788037040 con 0x7f379415f380 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.958+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37941f8be0 con 0x7f379415f380 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.959+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379406a360 con 0x7f379415f380 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.959+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3788044020 con 0x7f379415f380 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.960+0000 7f377bfff640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 0x7f378006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:38.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.960+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37880ac020 con 0x7f379415f380 2024-04-26T22:27:38.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.960+0000 7f3792ffd640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 0x7f378006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:38.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.960+0000 7f3792ffd640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 0x7f378006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37840011e0 tx=0x7f3784009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:38.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:38.964+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f378804b030 con 0x7f379415f380 2024-04-26T22:27:39.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.152+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f379415ecf0 con 0x7f379415f380 2024-04-26T22:27:39.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.153+0000 7f377bfff640 1 -- 172.21.15.62:0/1361927167 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f378807a4e0 con 0x7f379415f380 2024-04-26T22:27:39.154 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:39.154 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 msgr2=0x7f378006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 0x7f378006edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37840011e0 tx=0x7f3784009040 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 msgr2=0x7f3794155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f37880097d0 tx=0x7f37880046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 shutdown_connections 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f378006c8f0 0x7f378006edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f379415f760 0x7f3794155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f379415f380 0x7f3794155a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 --2- 172.21.15.62:0/1361927167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f37940d22e0 0x7f3794157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.155+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 >> 172.21.15.62:0/1361927167 conn(0x7f379409e560 msgr2=0x7f37940ba0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.156+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 shutdown_connections 2024-04-26T22:27:39.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.156+0000 7f3799a46640 1 -- 172.21.15.62:0/1361927167 wait complete. 2024-04-26T22:27:39.167 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:39.167 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:39.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.271+0000 7f1cc33bf640 1 Processor -- start 2024-04-26T22:27:39.354 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.271+0000 7f1cc33bf640 1 -- start start 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc33bf640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1cbc15f400 0x7f1cbc0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc33bf640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1cbc0b4cf0 0x7f1cbc1630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc33bf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cbc0b2760 con 0x7f1cbc0d5a80 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc33bf640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0b43e0 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc33bf640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1cbc0b2a00 con 0x7f1cbc15f400 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc195c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1cbc0b4cf0 0x7f1cbc1630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc095a640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1cbc0d5a80 0x7f1cbc0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39802/0 (socket says 172.21.15.62:39802) 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc095a640 1 -- 172.21.15.62:0/2952575194 learned_addr learned my addr 172.21.15.62:0/2952575194 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc115b640 1 --2- 172.21.15.62:0/2952575194 >> v2:172.21.15.174:3300/0 conn(0x7f1cbc15f400 0x7f1cbc0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc195c640 1 -- 172.21.15.62:0/2952575194 >> v2:172.21.15.174:3300/0 conn(0x7f1cbc15f400 msgr2=0x7f1cbc0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc195c640 1 --2- 172.21.15.62:0/2952575194 >> v2:172.21.15.174:3300/0 conn(0x7f1cbc15f400 0x7f1cbc0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc195c640 1 -- 172.21.15.62:0/2952575194 >> v1:172.21.15.62:6789/0 conn(0x7f1cbc0d5a80 legacy=0x7f1cbc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cc195c640 1 -- 172.21.15.62:0/2952575194 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc0b2ce0 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.272+0000 7f1cb3fff640 1 -- 172.21.15.62:0/2952575194 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256840575 0 0) 0x7f1cbc0b2760 con 0x7f1cbc0d5a80 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.273+0000 7f1cc195c640 1 --2- 172.21.15.62:0/2952575194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc1630b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06cc90 tx=0x7f1cb402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d988b4ebd88999 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.273+0000 7f1cb3fff640 1 -- 172.21.15.62:0/2952575194 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4007740 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.273+0000 7f1cb3fff640 1 -- 172.21.15.62:0/2952575194 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb40078e0 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.273+0000 7f1cb3fff640 1 -- 172.21.15.62:0/2952575194 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4007cb0 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 -- 172.21.15.62:0/2952575194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc0b4cf0 msgr2=0x7f1cbc1630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/2952575194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc1630b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc06cc90 tx=0x7f1cb402a5f0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 -- 172.21.15.62:0/2952575194 shutdown_connections 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/2952575194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc1630b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/2952575194 >> v2:172.21.15.174:3300/0 conn(0x7f1cbc15f400 0x7f1cbc0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 -- 172.21.15.62:0/2952575194 >> 172.21.15.62:0/2952575194 conn(0x7f1cbc09e4f0 msgr2=0x7f1cbc0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 -- 172.21.15.62:0/2952575194 shutdown_connections 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.274+0000 7f1cc33bf640 1 -- 172.21.15.62:0/2952575194 wait complete. 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.275+0000 7f1cc33bf640 1 Processor -- start 2024-04-26T22:27:39.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.275+0000 7f1cc33bf640 1 -- start start 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 0x7f1cbc161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0b3d00 con 0x7f1cbc15f400 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0b32a0 con 0x7f1cbc0b4cf0 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc33bf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0b3e80 con 0x7f1cbc0d5a80 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42376/0 (socket says 172.21.15.62:42376) 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 -- 172.21.15.62:0/3651084853 learned_addr learned my addr 172.21.15.62:0/3651084853 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc115b640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc095a640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 0x7f1cbc161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 -- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 msgr2=0x7f1cbc161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 0x7f1cbc161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 -- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cbc0b4cf0 msgr2=0x7f1cbc15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 -- 172.21.15.62:0/3651084853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc1600e0 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc095a640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 0x7f1cbc161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.276+0000 7f1cc195c640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009e80 tx=0x7f1cb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.277+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403f070 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.277+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb4031c40 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.277+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4009450 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.277+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1603c0 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.277+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc160c60 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.278+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cbc1607b0 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.278+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cb4003b90 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.279+0000 7f1cb1ffb640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 0x7f1ca006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.279+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb40acd10 con 0x7f1cbc15f400 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.279+0000 7f1cc115b640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 0x7f1ca006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.279+0000 7f1cc115b640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 0x7f1ca006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006fd0 tx=0x7f1cb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.362 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.283+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1cb407b140 con 0x7f1cbc15f400 2024-04-26T22:27:39.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.467+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1cbc160e30 con 0x7f1cbc15f400 2024-04-26T22:27:39.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.468+0000 7f1cb1ffb640 1 -- 172.21.15.62:0/3651084853 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1cb407aa40 con 0x7f1cbc15f400 2024-04-26T22:27:39.468 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.469+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 msgr2=0x7f1ca006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.469+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 0x7f1ca006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006fd0 tx=0x7f1cb8008040 comp rx=0 tx=0).stop 2024-04-26T22:27:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.469+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 msgr2=0x7f1cbc162080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.469+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009e80 tx=0x7f1cb4004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:39.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.469+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 shutdown_connections 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1ca006c8f0 0x7f1ca006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cbc15f400 0x7f1cbc162080 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1cbc0d5a80 0x7f1cbc161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 --2- 172.21.15.62:0/3651084853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cbc0b4cf0 0x7f1cbc15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 >> 172.21.15.62:0/3651084853 conn(0x7f1cbc09e4f0 msgr2=0x7f1cbc06df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 shutdown_connections 2024-04-26T22:27:39.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.470+0000 7f1cc33bf640 1 -- 172.21.15.62:0/3651084853 wait complete. 2024-04-26T22:27:39.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.586+0000 7f62eac2a640 1 Processor -- start 2024-04-26T22:27:39.586 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.586+0000 7f62eac2a640 1 -- start start 2024-04-26T22:27:39.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62eac2a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62eac2a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62eac2a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e40d2ed0 con 0x7f62e40d1fe0 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62eac2a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f62e40d1190 con 0x7f62e40d0d30 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62eac2a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f62e40d31b0 con 0x7f62e415f470 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e3fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e89c6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e91c7640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f62e40d1fe0 0x7f62e4162fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:36838/0 (socket says 172.21.15.62:36838) 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e3fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:47182/0 (socket says 172.21.15.62:47182) 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e91c7640 1 -- 172.21.15.62:0/3292106451 learned_addr learned my addr 172.21.15.62:0/3292106451 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.587+0000 7f62e89c6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:53124/0 (socket says 172.21.15.62:53124) 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e89c6640 1 -- 172.21.15.62:0/3292106451 >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 msgr2=0x7f62e40d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:39.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e89c6640 1 --2- 172.21.15.62:0/3292106451 >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e89c6640 1 -- 172.21.15.62:0/3292106451 >> v1:172.21.15.174:6789/0 conn(0x7f62e40d1fe0 legacy=0x7f62e4162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e89c6640 1 -- 172.21.15.62:0/3292106451 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e40d3490 con 0x7f62e415f470 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e3fff640 1 --2- 172.21.15.62:0/3292106451 >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.588+0000 7f62e89c6640 1 --2- 172.21.15.62:0/3292106451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097d0 tx=0x7f62d802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9040d77a6104586c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62e37fe640 1 -- 172.21.15.62:0/3292106451 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8037070 con 0x7f62e415f470 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62e37fe640 1 -- 172.21.15.62:0/3292106451 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d8032440 con 0x7f62e415f470 2024-04-26T22:27:39.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62e37fe640 1 -- 172.21.15.62:0/3292106451 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8032810 con 0x7f62e415f470 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 -- 172.21.15.62:0/3292106451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 msgr2=0x7f62e40d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 --2- 172.21.15.62:0/3292106451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f62d80097d0 tx=0x7f62d802a1c0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 -- 172.21.15.62:0/3292106451 shutdown_connections 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 --2- 172.21.15.62:0/3292106451 >> v2:172.21.15.174:3300/0 conn(0x7f62e40d0d30 0x7f62e40d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 --2- 172.21.15.62:0/3292106451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e415f470 0x7f62e40d07f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 -- 172.21.15.62:0/3292106451 >> 172.21.15.62:0/3292106451 conn(0x7f62e409e510 msgr2=0x7f62e40d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 -- 172.21.15.62:0/3292106451 shutdown_connections 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.589+0000 7f62eac2a640 1 -- 172.21.15.62:0/3292106451 wait complete. 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.590+0000 7f62eac2a640 1 Processor -- start 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.590+0000 7f62eac2a640 1 -- start start 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 0x7f62e4159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e40d1fe0 0x7f62e41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62e415f850 con 0x7f62e415f470 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62e40d3a50 con 0x7f62e40d0d30 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62eac2a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f62e40d3f70 con 0x7f62e40d1fe0 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62e91c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62e91c7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42384/0 (socket says 172.21.15.62:42384) 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62e89c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 0x7f62e4159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62e91c7640 1 -- 172.21.15.62:0/924362322 learned_addr learned my addr 172.21.15.62:0/924362322 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.591+0000 7f62e3fff640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e40d1fe0 0x7f62e41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 -- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e40d1fe0 msgr2=0x7f62e41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e40d1fe0 0x7f62e41541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 -- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 msgr2=0x7f62e4159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 0x7f62e4159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 -- 172.21.15.62:0/924362322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e41603a0 con 0x7f62e415f470 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e89c6640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 0x7f62e4159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e91c7640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00cab0 tx=0x7f62dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc017070 con 0x7f62e415f470 2024-04-26T22:27:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc00ac30 con 0x7f62e415f470 2024-04-26T22:27:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e4160680 con 0x7f62e415f470 2024-04-26T22:27:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc012400 con 0x7f62e415f470 2024-04-26T22:27:39.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.592+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e4154d80 con 0x7f62e415f470 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.593+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e406a360 con 0x7f62e415f470 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.593+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62dc01a070 con 0x7f62e415f470 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.594+0000 7f62e17fa640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 0x7f62c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.594+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62dc08b810 con 0x7f62e415f470 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.594+0000 7f62e89c6640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 0x7f62c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.595+0000 7f62e89c6640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 0x7f62c806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.598 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.598+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62dc059560 con 0x7f62e415f470 2024-04-26T22:27:39.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.785+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62e40d5870 con 0x7f62e415f470 2024-04-26T22:27:39.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.785+0000 7f62e17fa640 1 -- 172.21.15.62:0/924362322 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f62dc05d860 con 0x7f62e415f470 2024-04-26T22:27:39.786 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:39.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 msgr2=0x7f62c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 0x7f62c806ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62d8034000 tx=0x7f62d802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 msgr2=0x7f62e4154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00cab0 tx=0x7f62dc00cf70 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 shutdown_connections 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f62c806c920 0x7f62c806ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62e415f470 0x7f62e4154700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62e40d1fe0 0x7f62e41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 --2- 172.21.15.62:0/924362322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62e40d0d30 0x7f62e4159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 >> 172.21.15.62:0/924362322 conn(0x7f62e409e510 msgr2=0x7f62e40b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 shutdown_connections 2024-04-26T22:27:39.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.787+0000 7f62eac2a640 1 -- 172.21.15.62:0/924362322 wait complete. 2024-04-26T22:27:39.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.902+0000 7fc14374d640 1 Processor -- start 2024-04-26T22:27:39.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.902+0000 7fc14374d640 1 -- start start 2024-04-26T22:27:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc13c15f400 0x7fc13c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 0x7fc13c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc13c0b4b20 0x7fc13c0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c0b43e0 con 0x7fc13c15f400 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc13c0b2760 con 0x7fc13c0b4b20 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc14374d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c0b2a00 con 0x7fc13c0d5a80 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc13c15f400 0x7fc13c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc13c15f400 0x7fc13c0b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42386/0 (socket says 172.21.15.62:42386) 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 -- 172.21.15.62:0/2839250275 learned_addr learned my addr 172.21.15.62:0/2839250275 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc141cea640 1 --2- 172.21.15.62:0/2839250275 >> v2:172.21.15.174:3300/0 conn(0x7fc13c0b4b20 0x7fc13c0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc140ce8640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 0x7fc13c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 -- 172.21.15.62:0/2839250275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 msgr2=0x7fc13c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 0x7fc13c0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 -- 172.21.15.62:0/2839250275 >> v2:172.21.15.174:3300/0 conn(0x7fc13c0b4b20 msgr2=0x7fc13c0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 --2- 172.21.15.62:0/2839250275 >> v2:172.21.15.174:3300/0 conn(0x7fc13c0b4b20 0x7fc13c0d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc1414e9640 1 -- 172.21.15.62:0/2839250275 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c0b2ce0 con 0x7fc13c15f400 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.903+0000 7fc140ce8640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 0x7fc13c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:39.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.904+0000 7fc1414e9640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c15f400 0x7fc13c0b37c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc13c074870 tx=0x7fc13802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a0a9a49958238 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.904+0000 7fc133fff640 1 -- 172.21.15.62:0/2839250275 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138037070 con 0x7fc13c15f400 2024-04-26T22:27:39.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.904+0000 7fc133fff640 1 -- 172.21.15.62:0/2839250275 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc138032440 con 0x7fc13c15f400 2024-04-26T22:27:39.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.904+0000 7fc133fff640 1 -- 172.21.15.62:0/2839250275 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc138032810 con 0x7fc13c15f400 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 -- 172.21.15.62:0/2839250275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c15f400 msgr2=0x7fc13c0b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c15f400 0x7fc13c0b37c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc13c074870 tx=0x7fc13802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 -- 172.21.15.62:0/2839250275 shutdown_connections 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 --2- 172.21.15.62:0/2839250275 >> v2:172.21.15.174:3300/0 conn(0x7fc13c0b4b20 0x7fc13c0d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0d5a80 0x7fc13c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 --2- 172.21.15.62:0/2839250275 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c15f400 0x7fc13c0b37c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 -- 172.21.15.62:0/2839250275 >> 172.21.15.62:0/2839250275 conn(0x7fc13c09e4f0 msgr2=0x7fc13c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 -- 172.21.15.62:0/2839250275 shutdown_connections 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.905+0000 7fc14374d640 1 -- 172.21.15.62:0/2839250275 wait complete. 2024-04-26T22:27:39.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.906+0000 7fc14374d640 1 Processor -- start 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.906+0000 7fc14374d640 1 -- start start 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0b4b20 0x7fc13c0b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc13c15f400 0x7fc13c0b9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c0b4f70 con 0x7fc13c0d5a80 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c0b32a0 con 0x7fc13c15f400 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc14374d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc13c0b3d00 con 0x7fc13c0b4b20 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42392/0 (socket says 172.21.15.62:42392) 2024-04-26T22:27:39.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 -- 172.21.15.62:0/1072951261 learned_addr learned my addr 172.21.15.62:0/1072951261 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc1414e9640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0b4b20 0x7fc13c0b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc141cea640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc13c15f400 0x7fc13c0b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 -- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0b4b20 msgr2=0x7fc13c0b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0b4b20 0x7fc13c0b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 -- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc13c15f400 msgr2=0x7fc13c0b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc13c15f400 0x7fc13c0b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.907+0000 7fc140ce8640 1 -- 172.21.15.62:0/1072951261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c1554c0 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc140ce8640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c7f0 tx=0x7fc12c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c015070 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12c0077c0 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c0ba3d0 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c007b70 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.908+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c0ba990 con 0x7fc13c0d5a80 2024-04-26T22:27:39.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.909+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc13c06a360 con 0x7fc13c0d5a80 2024-04-26T22:27:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.909+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc12c018070 con 0x7fc13c0d5a80 2024-04-26T22:27:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.910+0000 7fc131ffb640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 0x7fc12006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.910+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc12c08bac0 con 0x7fc13c0d5a80 2024-04-26T22:27:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.910+0000 7fc1414e9640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 0x7fc12006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:39.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.911+0000 7fc1414e9640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 0x7fc12006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc138034000 tx=0x7fc13802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:39.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:39.914+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc12c0598a0 con 0x7fc13c0d5a80 2024-04-26T22:27:40.102 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.102+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc13c0d5860 con 0x7fc13c0d5a80 2024-04-26T22:27:40.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.102+0000 7fc131ffb640 1 -- 172.21.15.62:0/1072951261 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fc12c05dba0 con 0x7fc13c0d5a80 2024-04-26T22:27:40.103 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 msgr2=0x7fc12006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 0x7fc12006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc138034000 tx=0x7fc13802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 msgr2=0x7fc13c154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00c7f0 tx=0x7fc12c00ccb0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 shutdown_connections 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc12006c920 0x7fc12006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc13c15f400 0x7fc13c0b9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc13c0d5a80 0x7fc13c154f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 --2- 172.21.15.62:0/1072951261 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc13c0b4b20 0x7fc13c0b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.104+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 >> 172.21.15.62:0/1072951261 conn(0x7fc13c09e4f0 msgr2=0x7fc13c06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.105+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 shutdown_connections 2024-04-26T22:27:40.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.105+0000 7fc14374d640 1 -- 172.21.15.62:0/1072951261 wait complete. 2024-04-26T22:27:40.115 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2024-04-26T22:27:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.216+0000 7f421c9f7640 1 Processor -- start 2024-04-26T22:27:40.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.216+0000 7f421c9f7640 1 -- start start 2024-04-26T22:27:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4218162e40 0x7f42180d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42180cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f42180cfd70 0x7f42180ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f4218163220 con 0x7f4218162e40 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f42180d0eb0 con 0x7f42180cfd70 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421c9f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f42180d01a0 con 0x7f42180b2050 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4218162e40 0x7f42180d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4218162e40 0x7f42180d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42396/0 (socket says 172.21.15.62:42396) 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 -- 172.21.15.62:0/3790474063 learned_addr learned my addr 172.21.15.62:0/3790474063 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:40.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f4215d9b640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42180cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f4216d9d640 1 --2- 172.21.15.62:0/3790474063 >> v2:172.21.15.174:3300/0 conn(0x7f42180cfd70 0x7f42180ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 -- 172.21.15.62:0/3790474063 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 msgr2=0x7f42180cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42180cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 -- 172.21.15.62:0/3790474063 >> v2:172.21.15.174:3300/0 conn(0x7f42180cfd70 msgr2=0x7f42180ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 --2- 172.21.15.62:0/3790474063 >> v2:172.21.15.174:3300/0 conn(0x7f42180cfd70 0x7f42180ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.217+0000 7f421659c640 1 -- 172.21.15.62:0/3790474063 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42180b1d70 con 0x7f4218162e40 2024-04-26T22:27:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.218+0000 7f421659c640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4218162e40 0x7f42180d1340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4218072bb0 tx=0x7f420802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58b83881ea608d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:40.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.218+0000 7f421559a640 1 -- 172.21.15.62:0/3790474063 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208037070 con 0x7f4218162e40 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.218+0000 7f421559a640 1 -- 172.21.15.62:0/3790474063 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4208032440 con 0x7f4218162e40 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.218+0000 7f421559a640 1 -- 172.21.15.62:0/3790474063 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208032810 con 0x7f4218162e40 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 -- 172.21.15.62:0/3790474063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4218162e40 msgr2=0x7f42180d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4218162e40 0x7f42180d1340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4218072bb0 tx=0x7f420802a140 comp rx=0 tx=0).stop 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 -- 172.21.15.62:0/3790474063 shutdown_connections 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 --2- 172.21.15.62:0/3790474063 >> v2:172.21.15.174:3300/0 conn(0x7f42180cfd70 0x7f42180ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42180cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 --2- 172.21.15.62:0/3790474063 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4218162e40 0x7f42180d1340 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 -- 172.21.15.62:0/3790474063 >> 172.21.15.62:0/3790474063 conn(0x7f421809ba50 msgr2=0x7f421809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 -- 172.21.15.62:0/3790474063 shutdown_connections 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.219+0000 7f421c9f7640 1 -- 172.21.15.62:0/3790474063 wait complete. 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.220+0000 7f421c9f7640 1 Processor -- start 2024-04-26T22:27:40.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.220+0000 7f421c9f7640 1 -- start start 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42181602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 0x7f42180b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42180d1f20 con 0x7f42180cfd70 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42180d1c40 con 0x7f4218162e40 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421c9f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f42180b24b0 con 0x7f42180b2050 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f4215d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f4215d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42398/0 (socket says 172.21.15.62:42398) 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f4215d9b640 1 -- 172.21.15.62:0/1192597577 learned_addr learned my addr 172.21.15.62:0/1192597577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f421659c640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42181602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.221+0000 7f4216d9d640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 0x7f42180b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 -- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 msgr2=0x7f42181602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42181602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 -- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 msgr2=0x7f42180b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 0x7f42180b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 -- 172.21.15.62:0/1192597577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42180c6ab0 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4216d9d640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 0x7f42180b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4215d9b640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f420c00d800 tx=0x7f420c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c018030 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c0096a0 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42180c6d90 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c009870 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.222+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42180c7630 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.223+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421806a360 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.223+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f420c01b070 con 0x7f42180cfd70 2024-04-26T22:27:40.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.224+0000 7f4206ffd640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.224+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f420c08bcd0 con 0x7f42180cfd70 2024-04-26T22:27:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.224+0000 7f421659c640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:40.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.225+0000 7f421659c640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 0x7f41f806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4208034000 tx=0x7f420802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:40.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.228+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f420c05a190 con 0x7f42180cfd70 2024-04-26T22:27:40.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.482+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) -- 0x7f42180c6f60 con 0x7f42180cfd70 2024-04-26T22:27:40.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.937+0000 7f4206ffd640 1 -- 172.21.15.62:0/1192597577 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v45) ==== 184+0+0 (secure 0 0 0) 0x7f420c059a90 con 0x7f42180cfd70 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.939+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 msgr2=0x7f41f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 0x7f41f806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4208034000 tx=0x7f420802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 msgr2=0x7f42180b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f420c00d800 tx=0x7f420c00dcc0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 shutdown_connections 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f41f806c8f0 0x7f41f806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.940 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4218162e40 0x7f42180b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42180cfd70 0x7f42180b3a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 --2- 172.21.15.62:0/1192597577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42180b2050 0x7f42181602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 >> 172.21.15.62:0/1192597577 conn(0x7f421809ba50 msgr2=0x7f42180702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 shutdown_connections 2024-04-26T22:27:40.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:40.940+0000 7f421c9f7640 1 -- 172.21.15.62:0/1192597577 wait complete. 2024-04-26T22:27:40.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T22:27:41.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.060+0000 7ff74f2f1640 1 Processor -- start 2024-04-26T22:27:41.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.060+0000 7ff74f2f1640 1 -- start start 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff74815f470 0x7ff7480d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7480d0d30 0x7ff7480d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7480d1fe0 0x7ff748162fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff7480d1190 con 0x7ff7480d1fe0 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff7480d2ed0 con 0x7ff7480d0d30 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74f2f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff7480d31b0 con 0x7ff74815f470 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74d88e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7480d1fe0 0x7ff748162fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74d88e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff7480d1fe0 0x7ff748162fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42414/0 (socket says 172.21.15.62:42414) 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.061+0000 7ff74d88e640 1 -- 172.21.15.62:0/4085749742 learned_addr learned my addr 172.21.15.62:0/4085749742 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d08d640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff74815f470 0x7ff7480d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74c88c640 1 --2- 172.21.15.62:0/4085749742 >> v2:172.21.15.174:3300/0 conn(0x7ff7480d0d30 0x7ff7480d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 -- 172.21.15.62:0/4085749742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff74815f470 msgr2=0x7ff7480d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff74815f470 0x7ff7480d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 -- 172.21.15.62:0/4085749742 >> v2:172.21.15.174:3300/0 conn(0x7ff7480d0d30 msgr2=0x7ff7480d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 --2- 172.21.15.62:0/4085749742 >> v2:172.21.15.174:3300/0 conn(0x7ff7480d0d30 0x7ff7480d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 -- 172.21.15.62:0/4085749742 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7480d3490 con 0x7ff7480d1fe0 2024-04-26T22:27:41.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.062+0000 7ff74d88e640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff748162fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff738009e50 tx=0x7ff73802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79b041465e97b8b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff747fff640 1 -- 172.21.15.62:0/4085749742 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738036070 con 0x7ff7480d1fe0 2024-04-26T22:27:41.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff747fff640 1 -- 172.21.15.62:0/4085749742 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738031600 con 0x7ff7480d1fe0 2024-04-26T22:27:41.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff747fff640 1 -- 172.21.15.62:0/4085749742 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7380319d0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 -- 172.21.15.62:0/4085749742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 msgr2=0x7ff748162fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff748162fd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff738009e50 tx=0x7ff73802a6c0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 -- 172.21.15.62:0/4085749742 shutdown_connections 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff748162fd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/4085749742 >> v2:172.21.15.174:3300/0 conn(0x7ff7480d0d30 0x7ff7480d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/4085749742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff74815f470 0x7ff7480d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 -- 172.21.15.62:0/4085749742 >> 172.21.15.62:0/4085749742 conn(0x7ff74809e510 msgr2=0x7ff7480d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 -- 172.21.15.62:0/4085749742 shutdown_connections 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.063+0000 7ff74f2f1640 1 -- 172.21.15.62:0/4085749742 wait complete. 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.064+0000 7ff74f2f1640 1 Processor -- start 2024-04-26T22:27:41.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.064+0000 7ff74f2f1640 1 -- start start 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 0x7ff7480cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74815f470 0x7ff74815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff74815f850 con 0x7ff7480d1fe0 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7480d3a50 con 0x7ff74815f470 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74f2f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff7480d3f70 con 0x7ff7480d0d30 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74c88c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74c88c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42424/0 (socket says 172.21.15.62:42424) 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74c88c640 1 -- 172.21.15.62:0/1457179223 learned_addr learned my addr 172.21.15.62:0/1457179223 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74d88e640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74815f470 0x7ff74815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.065+0000 7ff74d08d640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 0x7ff7480cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 -- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 msgr2=0x7ff7480cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 0x7ff7480cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 -- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74815f470 msgr2=0x7ff74815dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74815f470 0x7ff74815dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 -- 172.21.15.62:0/1457179223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7481eafa0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74d08d640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 0x7ff7480cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74c88c640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00d5f0 tx=0x7ff73c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0125f0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c019070 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7481eb280 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0129d0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.066+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74815e530 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.067+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74806a360 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.067+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff73c012b70 con 0x7ff7480d1fe0 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.068+0000 7ff745ffb640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 0x7ff73406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.068+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff73c08be20 con 0x7ff7480d1fe0 2024-04-26T22:27:41.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.068+0000 7ff74d08d640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 0x7ff73406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.069+0000 7ff74d08d640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 0x7ff73406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b970 tx=0x7ff740008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.072+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff73c05a2e0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.261+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff7480b62e0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.264+0000 7ff745ffb640 1 -- 172.21.15.62:0/1457179223 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7ff73c059be0 con 0x7ff7480d1fe0 2024-04-26T22:27:41.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 msgr2=0x7ff73406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 0x7ff73406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b970 tx=0x7ff740008040 comp rx=0 tx=0).stop 2024-04-26T22:27:41.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 msgr2=0x7ff7481ea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00d5f0 tx=0x7ff73c00dab0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 shutdown_connections 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff73406c920 0x7ff73406ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff74815f470 0x7ff74815dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff7480d1fe0 0x7ff7481ea830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 --2- 172.21.15.62:0/1457179223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff7480d0d30 0x7ff7480cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 >> 172.21.15.62:0/1457179223 conn(0x7ff74809e510 msgr2=0x7ff74806d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 shutdown_connections 2024-04-26T22:27:41.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.266+0000 7ff74f2f1640 1 -- 172.21.15.62:0/1457179223 wait complete. 2024-04-26T22:27:41.277 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T22:27:41.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:27:41.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:27:41.357 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T22:27:41.357 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:41.358 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:41.374 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:41.374 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:41.429 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:41.429 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:41.484 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.624+0000 7f8a4e3af640 1 Processor -- start 2024-04-26T22:27:41.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.624+0000 7f8a4e3af640 1 -- start start 2024-04-26T22:27:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a4e3af640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a48162e40 0x7f8a480d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a4e3af640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a480b1d70 con 0x7f8a480b2050 2024-04-26T22:27:41.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a4e3af640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a480d1960 con 0x7f8a480cff40 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a4e3af640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8a48163220 con 0x7f8a48162e40 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a47fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8a48162e40 0x7f8a480d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a477fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8a480b2050 0x7f8a480cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39808/0 (socket says 172.21.15.62:39808) 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a477fe640 1 -- 172.21.15.62:0/1450232863 learned_addr learned my addr 172.21.15.62:0/1450232863 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a47fff640 1 -- 172.21.15.62:0/1450232863 >> v1:172.21.15.174:6789/0 conn(0x7f8a480cff40 legacy=0x7f8a480ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a47fff640 1 -- 172.21.15.62:0/1450232863 >> v1:172.21.15.62:6789/0 conn(0x7f8a480b2050 legacy=0x7f8a480cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a47fff640 1 -- 172.21.15.62:0/1450232863 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a480d1c40 con 0x7f8a48162e40 2024-04-26T22:27:41.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.625+0000 7f8a46ffd640 1 -- 172.21.15.62:0/1450232863 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026827479 0 0) 0x7f8a480b1d70 con 0x7f8a480b2050 2024-04-26T22:27:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.626+0000 7f8a47fff640 1 --2- 172.21.15.62:0/1450232863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a48162e40 0x7f8a480d1340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097a0 tx=0x7f8a3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f3a5730e1a054a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.626+0000 7f8a46ffd640 1 -- 172.21.15.62:0/1450232863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c037070 con 0x7f8a48162e40 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.626+0000 7f8a46ffd640 1 -- 172.21.15.62:0/1450232863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3c02ae50 con 0x7f8a48162e40 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.626+0000 7f8a46ffd640 1 -- 172.21.15.62:0/1450232863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c032660 con 0x7f8a48162e40 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 -- 172.21.15.62:0/1450232863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a48162e40 msgr2=0x7f8a480d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/1450232863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a48162e40 0x7f8a480d1340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097a0 tx=0x7f8a3c02a140 comp rx=0 tx=0).stop 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 -- 172.21.15.62:0/1450232863 shutdown_connections 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/1450232863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a48162e40 0x7f8a480d1340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 -- 172.21.15.62:0/1450232863 >> 172.21.15.62:0/1450232863 conn(0x7f8a4809ba50 msgr2=0x7f8a4809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 -- 172.21.15.62:0/1450232863 shutdown_connections 2024-04-26T22:27:41.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.627+0000 7f8a4e3af640 1 -- 172.21.15.62:0/1450232863 wait complete. 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.628+0000 7f8a4e3af640 1 Processor -- start 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.628+0000 7f8a4e3af640 1 -- start start 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a480cff40 0x7f8a4815c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 0x7f8a48071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a480b24e0 con 0x7f8a480b2050 2024-04-26T22:27:41.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a480d0eb0 con 0x7f8a480cff40 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4e3af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4806ea40 con 0x7f8a48162e40 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42446/0 (socket says 172.21.15.62:42446) 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 -- 172.21.15.62:0/271583606 learned_addr learned my addr 172.21.15.62:0/271583606 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a477fe640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a480cff40 0x7f8a4815c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4c94c640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 0x7f8a48071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 -- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 msgr2=0x7f8a48071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 0x7f8a48071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 -- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a480cff40 msgr2=0x7f8a4815c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a480cff40 0x7f8a4815c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 -- 172.21.15.62:0/271583606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a480cb0c0 con 0x7f8a480b2050 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a4c94c640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 0x7f8a48071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.629+0000 7f8a47fff640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097a0 tx=0x7f8a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.630+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c037040 con 0x7f8a480b2050 2024-04-26T22:27:41.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.630+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3c004030 con 0x7f8a480b2050 2024-04-26T22:27:41.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.630+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a480cc400 con 0x7f8a480b2050 2024-04-26T22:27:41.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.630+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c006e10 con 0x7f8a480b2050 2024-04-26T22:27:41.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.630+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a480cb640 con 0x7f8a480b2050 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.631+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4806a360 con 0x7f8a480b2050 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.631+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a3c03d650 con 0x7f8a480b2050 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.631+0000 7f8a44ff9640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 0x7f8a3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.632+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a3c0ac460 con 0x7f8a480b2050 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.632+0000 7f8a477fe640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 0x7f8a3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.632+0000 7f8a477fe640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 0x7f8a3406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a48071eb0 tx=0x7f8a38007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.636+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a3c042020 con 0x7f8a480b2050 2024-04-26T22:27:41.823 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.823+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a480cf9f0 con 0x7f8a480b2050 2024-04-26T22:27:41.824 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.824+0000 7f8a44ff9640 1 -- 172.21.15.62:0/271583606 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f8a3c07a890 con 0x7f8a480b2050 2024-04-26T22:27:41.824 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:41.824 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.825+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 msgr2=0x7f8a3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.825+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 0x7f8a3406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a48071eb0 tx=0x7f8a38007b50 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.825+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 msgr2=0x7f8a481618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.825+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0097a0 tx=0x7f8a3c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 shutdown_connections 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a3406c8f0 0x7f8a3406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a48162e40 0x7f8a48071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a480cff40 0x7f8a4815c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 --2- 172.21.15.62:0/271583606 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a480b2050 0x7f8a481618f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.826 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 >> 172.21.15.62:0/271583606 conn(0x7f8a4809ba50 msgr2=0x7f8a480cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 shutdown_connections 2024-04-26T22:27:41.827 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.826+0000 7f8a4e3af640 1 -- 172.21.15.62:0/271583606 wait complete. 2024-04-26T22:27:41.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:41.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.941+0000 7f59a5338640 1 Processor -- start 2024-04-26T22:27:41.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.941+0000 7f59a5338640 1 -- start start 2024-04-26T22:27:41.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f59a5338640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59a0145990 0x7f59a0148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f59a5338640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f59a5338640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a0145d70 con 0x7f59a006c780 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f59a5338640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f59a00703a0 con 0x7f59a0145990 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f59a5338640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00cbcc0 con 0x7f59a006f3e0 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599effd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f59a0145990 0x7f59a0148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599f7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f59a006c780 0x7f59a0162fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39824/0 (socket says 172.21.15.62:39824) 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599f7fe640 1 -- 172.21.15.62:0/4142951684 learned_addr learned my addr 172.21.15.62:0/4142951684 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f5997fff640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599effd640 1 -- 172.21.15.62:0/4142951684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 msgr2=0x7f59a006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599effd640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599effd640 1 -- 172.21.15.62:0/4142951684 >> v1:172.21.15.62:6789/0 conn(0x7f59a006c780 legacy=0x7f59a0162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599e7fc640 1 -- 172.21.15.62:0/4142951684 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3160736488 0 0) 0x7f59a0145d70 con 0x7f59a006c780 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f599effd640 1 -- 172.21.15.62:0/4142951684 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a00cbfa0 con 0x7f59a0145990 2024-04-26T22:27:41.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.942+0000 7f5997fff640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.943+0000 7f599effd640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a0148750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59900097a0 tx=0x7f599002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee615d76bbc267 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.943+0000 7f599e7fc640 1 -- 172.21.15.62:0/4142951684 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990037070 con 0x7f59a0145990 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.943+0000 7f599e7fc640 1 -- 172.21.15.62:0/4142951684 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599002adb0 con 0x7f59a0145990 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.943+0000 7f599e7fc640 1 -- 172.21.15.62:0/4142951684 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990032660 con 0x7f59a0145990 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 -- 172.21.15.62:0/4142951684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 msgr2=0x7f59a0148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a0148750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59900097a0 tx=0x7f599002a060 comp rx=0 tx=0).stop 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 -- 172.21.15.62:0/4142951684 shutdown_connections 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 --2- 172.21.15.62:0/4142951684 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a0148750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 -- 172.21.15.62:0/4142951684 >> 172.21.15.62:0/4142951684 conn(0x7f59a00d1e30 msgr2=0x7f59a00d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 -- 172.21.15.62:0/4142951684 shutdown_connections 2024-04-26T22:27:41.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.944+0000 7f59a5338640 1 -- 172.21.15.62:0/4142951684 wait complete. 2024-04-26T22:27:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.945+0000 7f59a5338640 1 Processor -- start 2024-04-26T22:27:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.945+0000 7f59a5338640 1 -- start start 2024-04-26T22:27:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a006c780 0x7f59a0142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a01ea560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f59a0148f60 con 0x7f59a0145990 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59a0070030 con 0x7f59a006c780 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00d1690 con 0x7f59a006f3e0 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42452/0 (socket says 172.21.15.62:42452) 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 -- 172.21.15.62:0/3684259336 learned_addr learned my addr 172.21.15.62:0/3684259336 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599effd640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a006c780 0x7f59a0142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f5997fff640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a01ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 -- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 msgr2=0x7f59a01ea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a01ea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 -- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a006c780 msgr2=0x7f59a0142d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a006c780 0x7f59a0142d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 -- 172.21.15.62:0/3684259336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a00b4d90 con 0x7f59a0145990 2024-04-26T22:27:41.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f599f7fe640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f599800c780 tx=0x7f599800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998018070 con 0x7f59a0145990 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599800ce30 con 0x7f59a0145990 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a00b5070 con 0x7f59a0145990 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800a840 con 0x7f59a0145990 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.946+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a00b5630 con 0x7f59a0145990 2024-04-26T22:27:41.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.947+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a00693e0 con 0x7f59a0145990 2024-04-26T22:27:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.948+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f599801b070 con 0x7f59a0145990 2024-04-26T22:27:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.948+0000 7f59977fe640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 0x7f598006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.948+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f599808afa0 con 0x7f59a0145990 2024-04-26T22:27:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.949+0000 7f599effd640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 0x7f598006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:41.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.949+0000 7f599effd640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 0x7f598006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5990034000 tx=0x7f599002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:41.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:41.953+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5998058e10 con 0x7f59a0145990 2024-04-26T22:27:42.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.138+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59a01e7bd0 con 0x7f59a0145990 2024-04-26T22:27:42.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.139+0000 7f59977fe640 1 -- 172.21.15.62:0/3684259336 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f599805d110 con 0x7f59a0145990 2024-04-26T22:27:42.139 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:42.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 msgr2=0x7f598006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 0x7f598006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5990034000 tx=0x7f599002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:42.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 msgr2=0x7f59a01e5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f599800c780 tx=0x7f599800cc40 comp rx=0 tx=0).stop 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 shutdown_connections 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f598006c8f0 0x7f598006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f59a0145990 0x7f59a01e5de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f59a006f3e0 0x7f59a01ea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 --2- 172.21.15.62:0/3684259336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59a006c780 0x7f59a0142d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 >> 172.21.15.62:0/3684259336 conn(0x7f59a00d1e30 msgr2=0x7f59a00cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 shutdown_connections 2024-04-26T22:27:42.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.141+0000 7f59a5338640 1 -- 172.21.15.62:0/3684259336 wait complete. 2024-04-26T22:27:42.152 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:42.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.256+0000 7f449354d640 1 Processor -- start 2024-04-26T22:27:42.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.256+0000 7f449354d640 1 -- start start 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.256+0000 7f449354d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c15f400 0x7f448c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f449354d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f448c0d5a80 0x7f448c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f449354d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f448c0b4b20 0x7f448c0d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f449354d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f448c0b43e0 con 0x7f448c0d5a80 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f449354d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f448c0b2760 con 0x7f448c0b4b20 2024-04-26T22:27:42.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f449354d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f448c0b2a00 con 0x7f448c15f400 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f448c0d5a80 0x7f448c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f448c0d5a80 0x7f448c0b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42466/0 (socket says 172.21.15.62:42466) 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 -- 172.21.15.62:0/2301420441 learned_addr learned my addr 172.21.15.62:0/2301420441 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f44912e9640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c15f400 0x7f448c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4491aea640 1 --2- 172.21.15.62:0/2301420441 >> v2:172.21.15.174:3300/0 conn(0x7f448c0b4b20 0x7f448c0d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 -- 172.21.15.62:0/2301420441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c15f400 msgr2=0x7f448c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c15f400 0x7f448c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 -- 172.21.15.62:0/2301420441 >> v2:172.21.15.174:3300/0 conn(0x7f448c0b4b20 msgr2=0x7f448c0d3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 --2- 172.21.15.62:0/2301420441 >> v2:172.21.15.174:3300/0 conn(0x7f448c0b4b20 0x7f448c0d3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.257+0000 7f4490ae8640 1 -- 172.21.15.62:0/2301420441 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c0b2ce0 con 0x7f448c0d5a80 2024-04-26T22:27:42.259 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f4490ae8640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0d5a80 0x7f448c0b45e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f448c074870 tx=0x7f448002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404b58d4243d71d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f448bfff640 1 -- 172.21.15.62:0/2301420441 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480037070 con 0x7f448c0d5a80 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f448bfff640 1 -- 172.21.15.62:0/2301420441 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4480032440 con 0x7f448c0d5a80 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f448bfff640 1 -- 172.21.15.62:0/2301420441 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480032810 con 0x7f448c0d5a80 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 -- 172.21.15.62:0/2301420441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0d5a80 msgr2=0x7f448c0b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0d5a80 0x7f448c0b45e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f448c074870 tx=0x7f448002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 -- 172.21.15.62:0/2301420441 shutdown_connections 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 --2- 172.21.15.62:0/2301420441 >> v2:172.21.15.174:3300/0 conn(0x7f448c0b4b20 0x7f448c0d3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0d5a80 0x7f448c0b45e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 --2- 172.21.15.62:0/2301420441 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c15f400 0x7f448c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 -- 172.21.15.62:0/2301420441 >> 172.21.15.62:0/2301420441 conn(0x7f448c09e4f0 msgr2=0x7f448c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.258+0000 7f449354d640 1 -- 172.21.15.62:0/2301420441 shutdown_connections 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.259+0000 7f449354d640 1 -- 172.21.15.62:0/2301420441 wait complete. 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 Processor -- start 2024-04-26T22:27:42.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 -- start start 2024-04-26T22:27:42.262 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 0x7f448c162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c15f400 0x7f448c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f448c0b4f70 con 0x7f448c0b4b20 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f448c0b32a0 con 0x7f448c15f400 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f449354d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f448c0b3d00 con 0x7f448c0d5a80 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f44912e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.260+0000 7f44912e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42476/0 (socket says 172.21.15.62:42476) 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 -- 172.21.15.62:0/1205220423 learned_addr learned my addr 172.21.15.62:0/1205220423 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4491aea640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c15f400 0x7f448c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4490ae8640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 0x7f448c162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 -- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 msgr2=0x7f448c162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 0x7f448c162190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 -- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c15f400 msgr2=0x7f448c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c15f400 0x7f448c0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 -- 172.21.15.62:0/1205220423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c160370 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4490ae8640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 0x7f448c162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f44912e9640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f448400d800 tx=0x7f448400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448400b660 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4484019070 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c160610 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448400ba20 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.261+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c160bd0 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.262+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c06a360 con 0x7f448c0b4b20 2024-04-26T22:27:42.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.263+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f448400bbc0 con 0x7f448c0b4b20 2024-04-26T22:27:42.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.263+0000 7f4489ffb640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 0x7f447806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.263+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f448408be70 con 0x7f448c0b4b20 2024-04-26T22:27:42.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.263+0000 7f4490ae8640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 0x7f447806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.264+0000 7f4490ae8640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 0x7f447806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4480034000 tx=0x7f448002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.268+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f448405a330 con 0x7f448c0b4b20 2024-04-26T22:27:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.451+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f448c0d28f0 con 0x7f448c0b4b20 2024-04-26T22:27:42.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.452+0000 7f4489ffb640 1 -- 172.21.15.62:0/1205220423 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4484059c30 con 0x7f448c0b4b20 2024-04-26T22:27:42.452 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 msgr2=0x7f447806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 0x7f447806edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4480034000 tx=0x7f448002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 msgr2=0x7f448c151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f448400d800 tx=0x7f448400dcc0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 shutdown_connections 2024-04-26T22:27:42.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f447806c8f0 0x7f447806edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c15f400 0x7f448c0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f448c0d5a80 0x7f448c162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 --2- 172.21.15.62:0/1205220423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f448c0b4b20 0x7f448c151650 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 >> 172.21.15.62:0/1205220423 conn(0x7f448c09e4f0 msgr2=0x7f448c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 shutdown_connections 2024-04-26T22:27:42.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.454+0000 7f449354d640 1 -- 172.21.15.62:0/1205220423 wait complete. 2024-04-26T22:27:42.465 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:42.569 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.569+0000 7feb0d67e640 1 Processor -- start 2024-04-26T22:27:42.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.569+0000 7feb0d67e640 1 -- start start 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb0d67e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7feb080b4cf0 0x7feb0816a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb0d67e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb080b2760 con 0x7feb0815f400 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb0d67e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb080b2a00 con 0x7feb080d5a80 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb0d67e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7feb080b43e0 con 0x7feb080b4cf0 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb06ffd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7feb0815f400 0x7feb080b37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:39840/0 (socket says 172.21.15.62:39840) 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb06ffd640 1 -- 172.21.15.62:0/1422510396 learned_addr learned my addr 172.21.15.62:0/1422510396 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202109430 0 0) 0x7feb080b2760 con 0x7feb0815f400 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feaf4003520 con 0x7feb0815f400 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.570+0000 7feb077fe640 1 --2- 172.21.15.62:0/1422510396 >> v2:172.21.15.174:3300/0 conn(0x7feb080b4cf0 0x7feb0816a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1039153950 0 0) 0x7feaf4003520 con 0x7feb0815f400 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 >> v2:172.21.15.174:3300/0 conn(0x7feb080b4cf0 msgr2=0x7feb0816a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 --2- 172.21.15.62:0/1422510396 >> v2:172.21.15.174:3300/0 conn(0x7feb080b4cf0 0x7feb0816a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 >> v1:172.21.15.174:6789/0 conn(0x7feb080d5a80 legacy=0x7feb080b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb080b2ce0 con 0x7feb0815f400 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7feb00001a10 con 0x7feb0815f400 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 >> v1:172.21.15.62:6789/0 conn(0x7feb0815f400 legacy=0x7feb080b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf40039b0 0x7feaf4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 0x7feaf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaf40051c0 0x7feaf400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feaf400c130 con 0x7feaf4004800 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feaf400c2e0 con 0x7feaf40051c0 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feaf400c4e0 con 0x7feaf40039b0 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 0x7feaf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb06ffd640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf40039b0 0x7feaf4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb077fe640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaf40051c0 0x7feaf400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 -- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf40039b0 msgr2=0x7feaf4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf40039b0 0x7feaf4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 -- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaf40051c0 msgr2=0x7feaf400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaf40051c0 0x7feaf400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb067fc640 1 -- 172.21.15.62:0/1422510396 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf4003520 con 0x7feaf4004800 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb06ffd640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf40039b0 0x7feaf4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb077fe640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaf40051c0 0x7feaf400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.571+0000 7feb077fe640 1 -- 172.21.15.62:0/1422510396 reap_dead start 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.572+0000 7feb067fc640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 0x7feaf4004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feaf0001750 tx=0x7feaf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f275639428220a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.572+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0001370 con 0x7feaf4004800 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.572+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf00052d0 con 0x7feaf4004800 2024-04-26T22:27:42.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.572+0000 7feb05ffb640 1 -- 172.21.15.62:0/1422510396 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002abc0 con 0x7feaf4004800 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 -- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 msgr2=0x7feaf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 0x7feaf4004c80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feaf0001750 tx=0x7feaf4005830 comp rx=0 tx=0).stop 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 -- 172.21.15.62:0/1422510396 shutdown_connections 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1422510396 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaf4004800 0x7feaf4004c80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 -- 172.21.15.62:0/1422510396 >> 172.21.15.62:0/1422510396 conn(0x7feb0809e4f0 msgr2=0x7feb080d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 -- 172.21.15.62:0/1422510396 shutdown_connections 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.573+0000 7feb0d67e640 1 -- 172.21.15.62:0/1422510396 wait complete. 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.574+0000 7feb0d67e640 1 Processor -- start 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.574+0000 7feb0d67e640 1 -- start start 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf4004800 0x7feb0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08160de0 0x7feb081f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb0815f7e0 con 0x7feb081f9110 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb080b32a0 con 0x7feb08160de0 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feb0815f960 con 0x7feaf4004800 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42488/0 (socket says 172.21.15.62:42488) 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 -- 172.21.15.62:0/1634845957 learned_addr learned my addr 172.21.15.62:0/1634845957 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb077fe640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08160de0 0x7feb081f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb06ffd640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf4004800 0x7feb0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 -- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf4004800 msgr2=0x7feb0806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf4004800 0x7feb0806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 -- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08160de0 msgr2=0x7feb081f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08160de0 0x7feb081f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 -- 172.21.15.62:0/1634845957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb081612f0 con 0x7feb081f9110 2024-04-26T22:27:42.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb067fc640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0037040 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0005480 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb081f8630 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0007a30 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.575+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb081615d0 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.577+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0806a360 con 0x7feb081f9110 2024-04-26T22:27:42.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.577+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaf0044020 con 0x7feb081f9110 2024-04-26T22:27:42.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.577+0000 7feaff7fe640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.577+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feaf00ab840 con 0x7feb081f9110 2024-04-26T22:27:42.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.578+0000 7feb06ffd640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.578+0000 7feb06ffd640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 0x7feae806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb080c8ac0 tx=0x7feb000021b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.582+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feaf0079be0 con 0x7feb081f9110 2024-04-26T22:27:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.765+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb080d28f0 con 0x7feb081f9110 2024-04-26T22:27:42.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.765+0000 7feaff7fe640 1 -- 172.21.15.62:0/1634845957 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7feaf00794e0 con 0x7feb081f9110 2024-04-26T22:27:42.765 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:42.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 msgr2=0x7feae806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 0x7feae806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb080c8ac0 tx=0x7feb000021b0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 msgr2=0x7feb081608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.767 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf0001120 comp rx=0 tx=0).stop 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 shutdown_connections 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feae806c920 0x7feae806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08160de0 0x7feb081f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb081f9110 0x7feb081608a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 --2- 172.21.15.62:0/1634845957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaf4004800 0x7feb0806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 >> 172.21.15.62:0/1634845957 conn(0x7feb0809e4f0 msgr2=0x7feb080b9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 shutdown_connections 2024-04-26T22:27:42.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.767+0000 7feb0d67e640 1 -- 172.21.15.62:0/1634845957 wait complete. 2024-04-26T22:27:42.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:27:42.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.880+0000 7fabb021d640 1 Processor -- start 2024-04-26T22:27:42.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 -- start start 2024-04-26T22:27:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7faba8162b40 0x7faba80d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7faba80d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.883 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba80d0210 con 0x7faba80d1850 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7faba80d41c0 con 0x7faba8162b40 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabb021d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7faba80d04f0 con 0x7faba80d3820 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.881+0000 7fabae7ba640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7faba80d1850 0x7faba80d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60780/0 (socket says 172.21.15.62:60780) 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabae7ba640 1 -- 172.21.15.62:0/3395741883 learned_addr learned my addr 172.21.15.62:0/3395741883 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 --2- 172.21.15.62:0/3395741883 >> v2:172.21.15.174:3300/0 conn(0x7faba8162b40 0x7faba80d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabad7b8640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7faba80d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035474840 0 0) 0x7faba80d0210 con 0x7faba80d1850 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab94003520 con 0x7faba80d1850 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 857949733 0 0) 0x7fab94003520 con 0x7faba80d1850 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 msgr2=0x7faba80d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7faba80d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 >> v2:172.21.15.174:3300/0 conn(0x7faba8162b40 msgr2=0x7faba80d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 --2- 172.21.15.62:0/3395741883 >> v2:172.21.15.174:3300/0 conn(0x7faba8162b40 0x7faba80d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba80d07d0 con 0x7faba80d1850 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7faba00019f0 con 0x7faba80d1850 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 >> v1:172.21.15.62:6789/0 conn(0x7faba80d1850 legacy=0x7faba80d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab940039b0 0x7fab94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab94004800 0x7fab94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7fab940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faba80d04f0 con 0x7fab940039b0 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab9400b9c0 con 0x7fab94004800 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fab9400bb40 con 0x7faba80d3820 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab940039b0 0x7fab94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabad7b8640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab94004800 0x7fab94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabae7ba640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7fab940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 -- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 msgr2=0x7fab940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7faba80d3820 0x7fab940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 -- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab94004800 msgr2=0x7fab94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab94004800 0x7fab94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabadfb9640 1 -- 172.21.15.62:0/3395741883 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab94003520 con 0x7fab940039b0 2024-04-26T22:27:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.882+0000 7fabad7b8640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab94004800 0x7fab94004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.883+0000 7fabadfb9640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab940039b0 0x7fab94003dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009ac0 tx=0x7fab94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e983d675f7348e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.883+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c036070 con 0x7fab940039b0 2024-04-26T22:27:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.883+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c031400 con 0x7fab940039b0 2024-04-26T22:27:42.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.883+0000 7fabacfb7640 1 -- 172.21.15.62:0/3395741883 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c009690 con 0x7fab940039b0 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 -- 172.21.15.62:0/3395741883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab940039b0 msgr2=0x7fab94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 --2- 172.21.15.62:0/3395741883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fab940039b0 0x7fab94003dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009ac0 tx=0x7fab94005830 comp rx=0 tx=0).stop 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabae7ba640 1 -- 172.21.15.62:0/3395741883 reap_dead start 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 -- 172.21.15.62:0/3395741883 shutdown_connections 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 -- 172.21.15.62:0/3395741883 >> 172.21.15.62:0/3395741883 conn(0x7faba809e550 msgr2=0x7faba80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 -- 172.21.15.62:0/3395741883 shutdown_connections 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.884+0000 7fabb021d640 1 -- 172.21.15.62:0/3395741883 wait complete. 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.885+0000 7fabb021d640 1 Processor -- start 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.885+0000 7fabb021d640 1 -- start start 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab940039b0 0x7faba80c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faba80ca330 0x7faba80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faba80b5bb0 con 0x7faba81f8230 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faba80d0d90 con 0x7faba80ca330 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabb021d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7faba80d6110 con 0x7fab940039b0 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42976/0 (socket says 172.21.15.62:42976) 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 -- 172.21.15.62:0/657444757 learned_addr learned my addr 172.21.15.62:0/657444757 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabadfb9640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab940039b0 0x7faba80c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabae7ba640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faba80ca330 0x7faba80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 -- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab940039b0 msgr2=0x7faba80c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab940039b0 0x7faba80c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 -- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faba80ca330 msgr2=0x7faba80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faba80ca330 0x7faba80ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 -- 172.21.15.62:0/657444757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba80cb190 con 0x7faba81f8230 2024-04-26T22:27:42.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.886+0000 7fabad7b8640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faba4011700 tx=0x7faba4011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.887+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba400f9c0 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.887+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400fb60 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.887+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba80cb470 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.887+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4015460 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.887+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba80cba30 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.888+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba806a360 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.888+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faba4015600 con 0x7faba81f8230 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.889+0000 7fab9a7fc640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 0x7fab8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:42.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.889+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faba408c4e0 con 0x7faba81f8230 2024-04-26T22:27:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.889+0000 7fabadfb9640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 0x7fab8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:42.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.890+0000 7fabadfb9640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 0x7fab8c06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab9c033000 tx=0x7fab9c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:42.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:42.893+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faba4019030 con 0x7faba81f8230 2024-04-26T22:27:43.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.093+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faba80d2960 con 0x7faba81f8230 2024-04-26T22:27:43.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.093+0000 7fab9a7fc640 1 -- 172.21.15.62:0/657444757 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7faba405a910 con 0x7faba81f8230 2024-04-26T22:27:43.093 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 msgr2=0x7fab8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 0x7fab8c06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab9c033000 tx=0x7fab9c008030 comp rx=0 tx=0).stop 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 msgr2=0x7faba81f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faba4011700 tx=0x7faba4011bc0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 shutdown_connections 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fab8c06c920 0x7fab8c06ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faba80ca330 0x7faba80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faba81f8230 0x7faba81f8690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 --2- 172.21.15.62:0/657444757 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab940039b0 0x7faba80c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 >> 172.21.15.62:0/657444757 conn(0x7faba809e550 msgr2=0x7faba80b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:43.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 shutdown_connections 2024-04-26T22:27:43.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.095+0000 7fabb021d640 1 -- 172.21.15.62:0/657444757 wait complete. 2024-04-26T22:27:43.106 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T22:27:43.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:43.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T22:27:43.131 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:27:43.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:43.131 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:27:43.195 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:43.195 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:43.214 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:43.214 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:43.272 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:43.272 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:43.330 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:43.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.435+0000 7f7050b04640 1 Processor -- start 2024-04-26T22:27:43.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.435+0000 7f7050b04640 1 -- start start 2024-04-26T22:27:43.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7050b04640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f704c162b40 0x7f704c0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7050b04640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f704c0d3820 0x7f704c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7050b04640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f704c0d0210 con 0x7f704c0d1850 2024-04-26T22:27:43.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7050b04640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f704c0d41c0 con 0x7f704c0d3820 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7050b04640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f704c0d04f0 con 0x7f704c162b40 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f704c0d3820 0x7f704c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f704c0d3820 0x7f704c0d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42990/0 (socket says 172.21.15.62:42990) 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 -- 172.21.15.62:0/3330841767 learned_addr learned my addr 172.21.15.62:0/3330841767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f704a59c640 1 --2- 172.21.15.62:0/3330841767 >> v2:172.21.15.174:3300/0 conn(0x7f704c162b40 0x7f704c0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 -- 172.21.15.62:0/3330841767 >> v2:172.21.15.174:3300/0 conn(0x7f704c162b40 msgr2=0x7f704c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 --2- 172.21.15.62:0/3330841767 >> v2:172.21.15.174:3300/0 conn(0x7f704c162b40 0x7f704c0d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 -- 172.21.15.62:0/3330841767 >> v1:172.21.15.62:6789/0 conn(0x7f704c0d1850 legacy=0x7f704c0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f704959a640 1 -- 172.21.15.62:0/3330841767 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764565377 0 0) 0x7f704c0d0210 con 0x7f704c0d1850 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.436+0000 7f7049d9b640 1 -- 172.21.15.62:0/3330841767 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c0d07d0 con 0x7f704c0d3820 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.437+0000 7f7049d9b640 1 --2- 172.21.15.62:0/3330841767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0d3c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d2910 tx=0x7f704002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213f2ca3701af584 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.437+0000 7f704959a640 1 -- 172.21.15.62:0/3330841767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040037070 con 0x7f704c0d3820 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.437+0000 7f704959a640 1 -- 172.21.15.62:0/3330841767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704002ae50 con 0x7f704c0d3820 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.437+0000 7f704959a640 1 -- 172.21.15.62:0/3330841767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040032860 con 0x7f704c0d3820 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 -- 172.21.15.62:0/3330841767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 msgr2=0x7f704c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 --2- 172.21.15.62:0/3330841767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0d3c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f704c0d2910 tx=0x7f704002a300 comp rx=0 tx=0).stop 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 -- 172.21.15.62:0/3330841767 shutdown_connections 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 --2- 172.21.15.62:0/3330841767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0d3c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 --2- 172.21.15.62:0/3330841767 >> v2:172.21.15.174:3300/0 conn(0x7f704c162b40 0x7f704c0d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 -- 172.21.15.62:0/3330841767 >> 172.21.15.62:0/3330841767 conn(0x7f704c09e550 msgr2=0x7f704c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 -- 172.21.15.62:0/3330841767 shutdown_connections 2024-04-26T22:27:43.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.438+0000 7f7050b04640 1 -- 172.21.15.62:0/3330841767 wait complete. 2024-04-26T22:27:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.439+0000 7f7050b04640 1 Processor -- start 2024-04-26T22:27:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.439+0000 7f7050b04640 1 -- start start 2024-04-26T22:27:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 0x7f704c1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 0x7f704c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0b5bb0 con 0x7f704c0d3820 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f704c0d0d90 con 0x7f704c162b40 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7050b04640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f704c0d6110 con 0x7f704c0d1850 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43000/0 (socket says 172.21.15.62:43000) 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 -- 172.21.15.62:0/1100129052 learned_addr learned my addr 172.21.15.62:0/1100129052 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f704ad9d640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 0x7f704c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f704a59c640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 0x7f704c1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 -- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 msgr2=0x7f704c1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 0x7f704c1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 -- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 msgr2=0x7f704c0ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.440+0000 7f7049d9b640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 0x7f704c0ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f7049d9b640 1 -- 172.21.15.62:0/1100129052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1f4a20 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f704a59c640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 0x7f704c1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f704ad9d640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 0x7f704c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f7049d9b640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f70400090b0 tx=0x7f70400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040037040 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7040004030 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c0cb6c0 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040004400 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.441+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c0b9f10 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.442+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c06a360 con 0x7f704c0d3820 2024-04-26T22:27:43.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.442+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f704003a070 con 0x7f704c0d3820 2024-04-26T22:27:43.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.443+0000 7f703affd640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 0x7f702c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.443+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f70400ac1e0 con 0x7f704c0d3820 2024-04-26T22:27:43.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.443+0000 7f704a59c640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 0x7f702c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.444+0000 7f704a59c640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 0x7f702c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f703c006fd0 tx=0x7f703c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.447+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f704c1f4a20 con 0x7f704c0d3820 2024-04-26T22:27:43.579 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:27:43.582 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:27:43.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.635+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f704c0d2960 con 0x7f704c0d3820 2024-04-26T22:27:43.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.636+0000 7f703affd640 1 -- 172.21.15.62:0/1100129052 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f7040041020 con 0x7f704c0d3820 2024-04-26T22:27:43.636 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:43.636 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 msgr2=0x7f702c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 0x7f702c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f703c006fd0 tx=0x7f703c008040 comp rx=0 tx=0).stop 2024-04-26T22:27:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 msgr2=0x7f704c0cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f70400090b0 tx=0x7f70400046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 shutdown_connections 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f702c06c8f0 0x7f702c06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f704c162b40 0x7f704c0ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f704c0d3820 0x7f704c0cb180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 --2- 172.21.15.62:0/1100129052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f704c0d1850 0x7f704c1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 >> 172.21.15.62:0/1100129052 conn(0x7f704c09e550 msgr2=0x7f704c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 shutdown_connections 2024-04-26T22:27:43.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.638+0000 7f7050b04640 1 -- 172.21.15.62:0/1100129052 wait complete. 2024-04-26T22:27:43.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:43.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:43.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.794+0000 7fcded31b640 1 Processor -- start 2024-04-26T22:27:43.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.794+0000 7fcded31b640 1 -- start start 2024-04-26T22:27:43.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcded31b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcde81610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcded31b640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde80d0930 con 0x7fcde80d5b60 2024-04-26T22:27:43.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcded31b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde80d0c00 con 0x7fcde80d5f80 2024-04-26T22:27:43.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcded31b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcde80d0ee0 con 0x7fcde8160800 2024-04-26T22:27:43.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcddffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fcde80d5b60 0x7fcde80d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60800/0 (socket says 172.21.15.62:60800) 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcddffff640 1 -- 172.21.15.62:0/516880950 learned_addr learned my addr 172.21.15.62:0/516880950 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcde6ffd640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcde81610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.795+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3338757576 0 0) 0x7fcde80d0930 con 0x7fcde80d5b60 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdd4003520 con 0x7fcde80d5b60 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846748159 0 0) 0x7fcde80d0c00 con 0x7fcde80d5f80 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcde80d0930 con 0x7fcde80d5f80 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 148142895 0 0) 0x7fcdd4003520 con 0x7fcde80d5b60 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 msgr2=0x7fcde81610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcde81610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 >> v1:172.21.15.174:6789/0 conn(0x7fcde80d5f80 legacy=0x7fcde80d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80d11c0 con 0x7fcde80d5b60 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fcdd0001670 con 0x7fcde80d5b60 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 >> v1:172.21.15.62:6789/0 conn(0x7fcde80d5b60 legacy=0x7fcde80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcdd4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdd4004120 0x7fcdd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd4004d40 0x7fcdd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80d0ee0 con 0x7fcdd4004d40 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd400bb00 con 0x7fcdd4004120 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd400bd00 con 0x7fcde8160800 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd4004d40 0x7fcdd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde6ffd640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcdd4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcddffff640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdd4004120 0x7fcdd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 -- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 msgr2=0x7fcdd4003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde8160800 0x7fcdd4003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 -- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdd4004120 msgr2=0x7fcdd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdd4004120 0x7fcdd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.796+0000 7fcde77fe640 1 -- 172.21.15.62:0/516880950 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4003520 con 0x7fcdd4004d40 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.797+0000 7fcde77fe640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd4004d40 0x7fcdd40051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001fd0 tx=0x7fcdd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26480e41f0208fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.797+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0001bf0 con 0x7fcdd4004d40 2024-04-26T22:27:43.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.797+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcde0003b70 con 0x7fcdd4004d40 2024-04-26T22:27:43.800 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.797+0000 7fcde67fc640 1 -- 172.21.15.62:0/516880950 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde00050d0 con 0x7fcdd4004d40 2024-04-26T22:27:43.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 -- 172.21.15.62:0/516880950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd4004d40 msgr2=0x7fcdd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 --2- 172.21.15.62:0/516880950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcdd4004d40 0x7fcdd40051c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001fd0 tx=0x7fcdd4005830 comp rx=0 tx=0).stop 2024-04-26T22:27:43.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcde77fe640 1 -- 172.21.15.62:0/516880950 reap_dead start 2024-04-26T22:27:43.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 -- 172.21.15.62:0/516880950 shutdown_connections 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 -- 172.21.15.62:0/516880950 >> 172.21.15.62:0/516880950 conn(0x7fcde809e550 msgr2=0x7fcde80d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 -- 172.21.15.62:0/516880950 shutdown_connections 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.798+0000 7fcded31b640 1 -- 172.21.15.62:0/516880950 wait complete. 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.799+0000 7fcded31b640 1 Processor -- start 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.799+0000 7fcded31b640 1 -- start start 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81546c0 0x7fcde80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde81f5660 0x7fcde80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80b6300 con 0x7fcde81f5280 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80d30f0 con 0x7fcde81546c0 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcded31b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcde8160be0 con 0x7fcde81f5660 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43018/0 (socket says 172.21.15.62:43018) 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 -- 172.21.15.62:0/2885651627 learned_addr learned my addr 172.21.15.62:0/2885651627 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:43.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde77fe640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde81f5660 0x7fcde80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcddffff640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81546c0 0x7fcde80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 -- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde81f5660 msgr2=0x7fcde80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde81f5660 0x7fcde80cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 -- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81546c0 msgr2=0x7fcde80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81546c0 0x7fcde80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.800+0000 7fcde6ffd640 1 -- 172.21.15.62:0/2885651627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80ca130 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcde6ffd640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80119a0 tx=0x7fcdd8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800faf0 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd800fc90 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde80ca410 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8014440 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.801+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde80ca9d0 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.802+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde80cb020 con 0x7fcde81f5280 2024-04-26T22:27:43.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.802+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdd8012040 con 0x7fcde81f5280 2024-04-26T22:27:43.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.803+0000 7fcddf7fe640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 0x7fcdc806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:43.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.803+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd808c240 con 0x7fcde81f5280 2024-04-26T22:27:43.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.803+0000 7fcddffff640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 0x7fcdc806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:43.805 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.804+0000 7fcddffff640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 0x7fcdc806ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ce210 tx=0x7fcdd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:43.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.807+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdd8019030 con 0x7fcde81f5280 2024-04-26T22:27:43.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.992+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcde80d4530 con 0x7fcde81f5280 2024-04-26T22:27:43.992 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.992+0000 7fcddf7fe640 1 -- 172.21.15.62:0/2885651627 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fcdd805a670 con 0x7fcde81f5280 2024-04-26T22:27:43.993 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 msgr2=0x7fcdc806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 0x7fcdc806ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ce210 tx=0x7fcdd0009040 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 msgr2=0x7fcde80ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80119a0 tx=0x7fcdd8011e60 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 shutdown_connections 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcdc806c920 0x7fcdc806ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcde81f5660 0x7fcde80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81546c0 0x7fcde80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 --2- 172.21.15.62:0/2885651627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcde81f5280 0x7fcde80ce2e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 >> 172.21.15.62:0/2885651627 conn(0x7fcde809e550 msgr2=0x7fcde80b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 shutdown_connections 2024-04-26T22:27:43.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:43.994+0000 7fcded31b640 1 -- 172.21.15.62:0/2885651627 wait complete. 2024-04-26T22:27:44.006 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:44.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.109+0000 7f99ca546640 1 Processor -- start 2024-04-26T22:27:44.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.109+0000 7f99ca546640 1 -- start start 2024-04-26T22:27:44.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99ca546640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f99c406e470 0x7f99c40ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.110 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99ca546640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c40d25f0 con 0x7f99c415f5e0 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99ca546640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c40d2030 con 0x7f99c40d2b30 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99ca546640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f99c40cda70 con 0x7f99c406e470 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f99c415f5e0 0x7f99c406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60808/0 (socket says 172.21.15.62:60808) 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c3fff640 1 -- 172.21.15.62:0/1346498298 learned_addr learned my addr 172.21.15.62:0/1346498298 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c8ae3640 1 --2- 172.21.15.62:0/1346498298 >> v2:172.21.15.174:3300/0 conn(0x7f99c406e470 0x7f99c40ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167560268 0 0) 0x7f99c40d25f0 con 0x7f99c415f5e0 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99ac003520 con 0x7f99c415f5e0 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68158935 0 0) 0x7f99c40d2030 con 0x7f99c40d2b30 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99c40d25f0 con 0x7f99c40d2b30 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c8ae3640 1 -- 172.21.15.62:0/1346498298 >> v1:172.21.15.174:6789/0 conn(0x7f99c40d2b30 legacy=0x7f99c40cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c8ae3640 1 -- 172.21.15.62:0/1346498298 >> v1:172.21.15.62:6789/0 conn(0x7f99c415f5e0 legacy=0x7f99c406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.111 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.110+0000 7f99c8ae3640 1 -- 172.21.15.62:0/1346498298 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c40d2310 con 0x7f99c406e470 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.111+0000 7f99c8ae3640 1 --2- 172.21.15.62:0/1346498298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c406e470 0x7f99c40ce610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009d90 tx=0x7f99bc02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80e7a1bdcaf0e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007740 con 0x7f99c406e470 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99bc0078e0 con 0x7f99c406e470 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99c2ffd640 1 -- 172.21.15.62:0/1346498298 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc007cb0 con 0x7f99c406e470 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 -- 172.21.15.62:0/1346498298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c406e470 msgr2=0x7f99c40ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 --2- 172.21.15.62:0/1346498298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c406e470 0x7f99c40ce610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009d90 tx=0x7f99bc02a660 comp rx=0 tx=0).stop 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 -- 172.21.15.62:0/1346498298 shutdown_connections 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 --2- 172.21.15.62:0/1346498298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c406e470 0x7f99c40ce610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 -- 172.21.15.62:0/1346498298 >> 172.21.15.62:0/1346498298 conn(0x7f99c40d2f80 msgr2=0x7f99c40d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 -- 172.21.15.62:0/1346498298 shutdown_connections 2024-04-26T22:27:44.113 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.112+0000 7f99ca546640 1 -- 172.21.15.62:0/1346498298 wait complete. 2024-04-26T22:27:44.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.113+0000 7f99ca546640 1 Processor -- start 2024-04-26T22:27:44.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 -- start start 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99c406e470 0x7f99c40cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 0x7f99c41502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f99c406d560 con 0x7f99c40d2b30 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99c40d1cb0 con 0x7f99c415f5e0 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99ca546640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f99c406d6e0 con 0x7f99c406e470 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.114+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.116 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43032/0 (socket says 172.21.15.62:43032) 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 -- 172.21.15.62:0/13135324 learned_addr learned my addr 172.21.15.62:0/13135324 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c8ae3640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 0x7f99c41502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c3fff640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99c406e470 0x7f99c40cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 -- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99c406e470 msgr2=0x7f99c40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99c406e470 0x7f99c40cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 -- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 msgr2=0x7f99c41502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 0x7f99c41502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 -- 172.21.15.62:0/13135324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c415fa60 con 0x7f99c40d2b30 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c8ae3640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 0x7f99c41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c37fe640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c490 tx=0x7f99b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b80052d0 con 0x7f99c40d2b30 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b8005470 con 0x7f99c40d2b30 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c415fd40 con 0x7f99c40d2b30 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8012850 con 0x7f99c40d2b30 2024-04-26T22:27:44.117 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.115+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c40cc9e0 con 0x7f99c40d2b30 2024-04-26T22:27:44.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.116+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c40693e0 con 0x7f99c40d2b30 2024-04-26T22:27:44.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.117+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99b80129f0 con 0x7f99c40d2b30 2024-04-26T22:27:44.118 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.118+0000 7f99c0ff9640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.118+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99b808b820 con 0x7f99c40d2b30 2024-04-26T22:27:44.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.118+0000 7f99c3fff640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.119 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.118+0000 7f99c3fff640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 0x7f99b006ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f99b40011e0 tx=0x7f99b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.122+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f99b8059c50 con 0x7f99c40d2b30 2024-04-26T22:27:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.307+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f99c40cf3b0 con 0x7f99c40d2b30 2024-04-26T22:27:44.307 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.307+0000 7f99c0ff9640 1 -- 172.21.15.62:0/13135324 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f99b8059550 con 0x7f99c40d2b30 2024-04-26T22:27:44.307 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 msgr2=0x7f99b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 0x7f99b006ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f99b40011e0 tx=0x7f99b4009040 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 msgr2=0x7f99c4161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c490 tx=0x7f99b800c950 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 shutdown_connections 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f99b006c920 0x7f99b006ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99c415f5e0 0x7f99c41502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f99c40d2b30 0x7f99c4161c60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 --2- 172.21.15.62:0/13135324 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f99c406e470 0x7f99c40cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.310 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.309+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 >> 172.21.15.62:0/13135324 conn(0x7f99c40d2f80 msgr2=0x7f99c40afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.310+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 shutdown_connections 2024-04-26T22:27:44.311 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.310+0000 7f99ca546640 1 -- 172.21.15.62:0/13135324 wait complete. 2024-04-26T22:27:44.321 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:44.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.425+0000 7fae2eb38640 1 Processor -- start 2024-04-26T22:27:44.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.425+0000 7fae2eb38640 1 -- start start 2024-04-26T22:27:44.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2eb38640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2eb38640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fae280d1fe0 0x7fae280b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2eb38640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae280d2ed0 con 0x7fae2815f470 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2eb38640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fae280d31b0 con 0x7fae280d1fe0 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2eb38640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fae280d3490 con 0x7fae280d0f00 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae27fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fae280d1fe0 0x7fae280b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2c8d4640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fae2815f470 0x7fae280d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:34344/0 (socket says 172.21.15.62:34344) 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2c8d4640 1 -- 172.21.15.62:0/3822127592 learned_addr learned my addr 172.21.15.62:0/3822127592 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fae280d1fe0 0x7fae280b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:44868/0 (socket says 172.21.15.62:44868) 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 -- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 msgr2=0x7fae280d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 --2- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 -- 172.21.15.62:0/3822127592 >> v1:172.21.15.174:6789/0 conn(0x7fae2815f470 legacy=0x7fae280d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae277fe640 1 -- 172.21.15.62:0/3822127592 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4056828646 0 0) 0x7fae280d2ed0 con 0x7fae2815f470 2024-04-26T22:27:44.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.426+0000 7fae2d0d5640 1 -- 172.21.15.62:0/3822127592 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae280d3770 con 0x7fae280d1fe0 2024-04-26T22:27:44.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.427+0000 7fae2d0d5640 1 --2- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae280b7fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae20009d90 tx=0x7fae2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78de2b4fa064395a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.427+0000 7fae277fe640 1 -- 172.21.15.62:0/3822127592 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20007740 con 0x7fae280d1fe0 2024-04-26T22:27:44.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.427+0000 7fae277fe640 1 -- 172.21.15.62:0/3822127592 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae200078e0 con 0x7fae280d1fe0 2024-04-26T22:27:44.429 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.427+0000 7fae277fe640 1 -- 172.21.15.62:0/3822127592 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20007cb0 con 0x7fae280d1fe0 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 -- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 msgr2=0x7fae280b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 --2- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae280b7fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae20009d90 tx=0x7fae2002a660 comp rx=0 tx=0).stop 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 -- 172.21.15.62:0/3822127592 shutdown_connections 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 --2- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae280b7fb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 --2- 172.21.15.62:0/3822127592 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 -- 172.21.15.62:0/3822127592 >> 172.21.15.62:0/3822127592 conn(0x7fae2809e510 msgr2=0x7fae280d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 -- 172.21.15.62:0/3822127592 shutdown_connections 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.428+0000 7fae2eb38640 1 -- 172.21.15.62:0/3822127592 wait complete. 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.429+0000 7fae2eb38640 1 Processor -- start 2024-04-26T22:27:44.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.429+0000 7fae2eb38640 1 -- start start 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae2814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fae280d3f70 con 0x7fae2815f470 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae2815f850 con 0x7fae280d1fe0 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2eb38640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fae2814cab0 con 0x7fae280d0f00 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2d0d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2d0d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43038/0 (socket says 172.21.15.62:43038) 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2d0d5640 1 -- 172.21.15.62:0/621925590 learned_addr learned my addr 172.21.15.62:0/621925590 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae27fff640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae2814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2c8d4640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.430+0000 7fae2d0d5640 1 -- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 msgr2=0x7fae280cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2d0d5640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2d0d5640 1 -- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 msgr2=0x7fae2814c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2d0d5640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae2814c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2d0d5640 1 -- 172.21.15.62:0/621925590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae280b7d90 con 0x7fae2815f470 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae27fff640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae2814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2d0d5640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae20009eb0 tx=0x7fae20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20040070 con 0x7fae2815f470 2024-04-26T22:27:44.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae20031bc0 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae280b8030 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20009450 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.431+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae280b85f0 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.432+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2806a360 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.432+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae20039040 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.433+0000 7fae257fa640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 0x7fae0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.433+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae200acb80 con 0x7fae2815f470 2024-04-26T22:27:44.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.433+0000 7fae2c8d4640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 0x7fae0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.436 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.433+0000 7fae2c8d4640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 0x7fae0c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f60 tx=0x7fae1c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.437+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae2007a9f0 con 0x7fae2815f470 2024-04-26T22:27:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.622+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae280d5870 con 0x7fae2815f470 2024-04-26T22:27:44.622 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.622+0000 7fae257fa640 1 -- 172.21.15.62:0/621925590 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fae2007ecf0 con 0x7fae2815f470 2024-04-26T22:27:44.623 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 msgr2=0x7fae0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 0x7fae0c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae1c000f60 tx=0x7fae1c007d50 comp rx=0 tx=0).stop 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 msgr2=0x7fae280d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae20009eb0 tx=0x7fae20004a40 comp rx=0 tx=0).stop 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 shutdown_connections 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fae0c06c8f0 0x7fae0c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fae2815f470 0x7fae280d0330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae280d1fe0 0x7fae2814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 --2- 172.21.15.62:0/621925590 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fae280d0f00 0x7fae280cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 >> 172.21.15.62:0/621925590 conn(0x7fae2809e510 msgr2=0x7fae280d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 shutdown_connections 2024-04-26T22:27:44.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.625+0000 7fae2eb38640 1 -- 172.21.15.62:0/621925590 wait complete. 2024-04-26T22:27:44.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:27:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.737+0000 7f4419a74640 1 Processor -- start 2024-04-26T22:27:44.737 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.737+0000 7f4419a74640 1 -- start start 2024-04-26T22:27:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4419a74640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f441406ea40 0x7f44140d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4419a74640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 0x7f44140cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4419a74640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44140b1d70 con 0x7f4414162e40 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4419a74640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4414163220 con 0x7f441406ea40 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4419a74640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44140d01c0 con 0x7f44140cf830 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f44137fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4414162e40 0x7f44140b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60820/0 (socket says 172.21.15.62:60820) 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f44137fe640 1 -- 172.21.15.62:0/3536237223 learned_addr learned my addr 172.21.15.62:0/3536237223 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4413fff640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 0x7f44140cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4412ffd640 1 --2- 172.21.15.62:0/3536237223 >> v2:172.21.15.174:3300/0 conn(0x7f441406ea40 0x7f44140d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046545098 0 0) 0x7f44140b1d70 con 0x7f4414162e40 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43fc003520 con 0x7f4414162e40 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4412ffd640 1 -- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 msgr2=0x7f44140cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:44.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4202870129 0 0) 0x7f43fc003520 con 0x7f4414162e40 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4412ffd640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 0x7f44140cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4412ffd640 1 -- 172.21.15.62:0/3536237223 >> v1:172.21.15.62:6789/0 conn(0x7f4414162e40 legacy=0x7f44140b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4412ffd640 1 -- 172.21.15.62:0/3536237223 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140d1960 con 0x7f441406ea40 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.738+0000 7f4413fff640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 0x7f44140cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.739+0000 7f4412ffd640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441406ea40 0x7f44140d1340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44080097a0 tx=0x7f440802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ee7c0f6a6e93d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.739+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408037070 con 0x7f441406ea40 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.739+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440802ab10 con 0x7f441406ea40 2024-04-26T22:27:44.741 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.739+0000 7f44127fc640 1 -- 172.21.15.62:0/3536237223 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404001670 con 0x7f441406ea40 2024-04-26T22:27:44.744 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 -- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441406ea40 msgr2=0x7f44140d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441406ea40 0x7f44140d1340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f44080097a0 tx=0x7f440802a080 comp rx=0 tx=0).stop 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 -- 172.21.15.62:0/3536237223 shutdown_connections 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f441406ea40 0x7f44140d1340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 --2- 172.21.15.62:0/3536237223 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44140cf830 0x7f44140cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 -- 172.21.15.62:0/3536237223 >> 172.21.15.62:0/3536237223 conn(0x7f441409ba50 msgr2=0x7f441409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 -- 172.21.15.62:0/3536237223 shutdown_connections 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.740+0000 7f4419a74640 1 -- 172.21.15.62:0/3536237223 wait complete. 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.741+0000 7f4419a74640 1 Processor -- start 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.741+0000 7f4419a74640 1 -- start start 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f441406ea40 0x7f44141f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44140cf830 0x7f441415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.745 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d1f20 con 0x7f4414162e40 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44140d0eb0 con 0x7f44140cf830 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4419a74640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44140ce010 con 0x7f441406ea40 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4413fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4413fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43042/0 (socket says 172.21.15.62:43042) 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4413fff640 1 -- 172.21.15.62:0/3293143152 learned_addr learned my addr 172.21.15.62:0/3293143152 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f44137fe640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f441406ea40 0x7f44141f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.742+0000 7f4412ffd640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44140cf830 0x7f441415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 -- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f441406ea40 msgr2=0x7f44141f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f441406ea40 0x7f44141f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 -- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44140cf830 msgr2=0x7f441415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44140cf830 0x7f441415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 -- 172.21.15.62:0/3293143152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4414070b20 con 0x7f4414162e40 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4413fff640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f440c00efc0 tx=0x7f440c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c018030 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c00cde0 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003bb0 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4414070e00 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.743+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44140713c0 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.744+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441406a360 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.744+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f440c01b070 con 0x7f4414162e40 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.745+0000 7f43fbfff640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 0x7f440006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:44.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.745+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f440c08c4b0 con 0x7f4414162e40 2024-04-26T22:27:44.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.745+0000 7f44137fe640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 0x7f440006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:44.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.746+0000 7f44137fe640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 0x7f440006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4404005880 tx=0x7f44040057f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:44.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.749+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f440c010040 con 0x7f4414162e40 2024-04-26T22:27:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.950+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f44140b1ce0 con 0x7f4414162e40 2024-04-26T22:27:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.951+0000 7f43fbfff640 1 -- 172.21.15.62:0/3293143152 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f440c05a290 con 0x7f4414162e40 2024-04-26T22:27:44.951 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 msgr2=0x7f440006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 0x7f440006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4404005880 tx=0x7f44040057f0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 msgr2=0x7f44140702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f440c00efc0 tx=0x7f440c009f90 comp rx=0 tx=0).stop 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 shutdown_connections 2024-04-26T22:27:44.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f440006c8f0 0x7f440006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4414162e40 0x7f44140702d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44140cf830 0x7f441415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 --2- 172.21.15.62:0/3293143152 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f441406ea40 0x7f44141f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 >> 172.21.15.62:0/3293143152 conn(0x7f441409ba50 msgr2=0x7f44140b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 shutdown_connections 2024-04-26T22:27:44.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:44.953+0000 7f4419a74640 1 -- 172.21.15.62:0/3293143152 wait complete. 2024-04-26T22:27:44.964 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T22:27:44.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:44.964 DEBUG:teuthology.orchestra.run.smithi174:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T22:27:44.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:27:44.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:44.995 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T22:27:45.062 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:45.062 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:45.078 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:45.078 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:45.133 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:27:45.133 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:45.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:45.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.327+0000 7f8dd6d89640 1 Processor -- start 2024-04-26T22:27:45.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.327+0000 7f8dd6d89640 1 -- start start 2024-04-26T22:27:45.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd015f380 0x7f8dd00d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8dd015f760 0x7f8dd00d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 0x7f8dd00cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00d43a0 con 0x7f8dd015f380 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd00d3de0 con 0x7f8dd015f760 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd6d89640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd00d40c0 con 0x7f8dd00d5b50 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd015f380 0x7f8dd00d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd015f380 0x7f8dd00d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43050/0 (socket says 172.21.15.62:43050) 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 -- 172.21.15.62:0/2286461125 learned_addr learned my addr 172.21.15.62:0/2286461125 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dcffff640 1 --2- 172.21.15.62:0/2286461125 >> v2:172.21.15.174:3300/0 conn(0x7f8dd015f760 0x7f8dd00d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd5326640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 0x7f8dd00cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 -- 172.21.15.62:0/2286461125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 msgr2=0x7f8dd00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 0x7f8dd00cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 -- 172.21.15.62:0/2286461125 >> v2:172.21.15.174:3300/0 conn(0x7f8dd015f760 msgr2=0x7f8dd00d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/2286461125 >> v2:172.21.15.174:3300/0 conn(0x7f8dd015f760 0x7f8dd00d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd4b25640 1 -- 172.21.15.62:0/2286461125 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd00b8280 con 0x7f8dd015f380 2024-04-26T22:27:45.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.328+0000 7f8dd5326640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 0x7f8dd00cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.329+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00d1bd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dd008d560 tx=0x7f8dc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d2e42447b70026 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.329+0000 7f8dcf7fe640 1 -- 172.21.15.62:0/2286461125 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4037070 con 0x7f8dd015f380 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.329+0000 7f8dcf7fe640 1 -- 172.21.15.62:0/2286461125 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc4032440 con 0x7f8dd015f380 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.329+0000 7f8dcf7fe640 1 -- 172.21.15.62:0/2286461125 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4032810 con 0x7f8dd015f380 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 -- 172.21.15.62:0/2286461125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 msgr2=0x7f8dd00d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00d1bd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8dd008d560 tx=0x7f8dc402a270 comp rx=0 tx=0).stop 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 -- 172.21.15.62:0/2286461125 shutdown_connections 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd00d5b50 0x7f8dd00cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/2286461125 >> v2:172.21.15.174:3300/0 conn(0x7f8dd015f760 0x7f8dd00d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/2286461125 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00d1bd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 -- 172.21.15.62:0/2286461125 >> 172.21.15.62:0/2286461125 conn(0x7f8dd009e560 msgr2=0x7f8dd00d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 -- 172.21.15.62:0/2286461125 shutdown_connections 2024-04-26T22:27:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.330+0000 7f8dd6d89640 1 -- 172.21.15.62:0/2286461125 wait complete. 2024-04-26T22:27:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.331+0000 7f8dd6d89640 1 Processor -- start 2024-04-26T22:27:45.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.331+0000 7f8dd6d89640 1 -- start start 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd00d5b50 0x7f8dd009efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd015f760 0x7f8dd01f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00d1280 con 0x7f8dd015f380 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd00d1100 con 0x7f8dd00d5b50 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd00d3840 con 0x7f8dd015f760 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43060/0 (socket says 172.21.15.62:43060) 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 -- 172.21.15.62:0/3764121986 learned_addr learned my addr 172.21.15.62:0/3764121986 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd5326640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd015f760 0x7f8dd01f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd00d5b50 0x7f8dd009efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 -- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd015f760 msgr2=0x7f8dd01f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd015f760 0x7f8dd01f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 -- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd00d5b50 msgr2=0x7f8dd009efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd00d5b50 0x7f8dd009efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 -- 172.21.15.62:0/3764121986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd01f5cd0 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcffff640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d600 tx=0x7f8dbc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc004240 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0043e0 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd01f6040 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.332+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc003830 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.333+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd01f6600 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.334+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd006a360 con 0x7f8dd015f380 2024-04-26T22:27:45.336 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.334+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dbc0039d0 con 0x7f8dd015f380 2024-04-26T22:27:45.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.335+0000 7f8dcd7fa640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.335+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8dbc08bdd0 con 0x7f8dd015f380 2024-04-26T22:27:45.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.335+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.337 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.336+0000 7f8dd4b25640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4034000 tx=0x7f8dc402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.340 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.339+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dbc05a290 con 0x7f8dd015f380 2024-04-26T22:27:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.525+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dd009f5b0 con 0x7f8dd015f380 2024-04-26T22:27:45.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.526+0000 7f8dcd7fa640 1 -- 172.21.15.62:0/3764121986 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f8dbc08d020 con 0x7f8dd015f380 2024-04-26T22:27:45.527 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:45.527 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 msgr2=0x7f8db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4034000 tx=0x7f8dc402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 msgr2=0x7f8dd00cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00d600 tx=0x7f8dbc00dac0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 shutdown_connections 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd015f760 0x7f8dd01f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd015f380 0x7f8dd00cb230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 --2- 172.21.15.62:0/3764121986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd00d5b50 0x7f8dd009efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 >> 172.21.15.62:0/3764121986 conn(0x7f8dd009e560 msgr2=0x7f8dd006e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 shutdown_connections 2024-04-26T22:27:45.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.528+0000 7f8dd6d89640 1 -- 172.21.15.62:0/3764121986 wait complete. 2024-04-26T22:27:45.540 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:45.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:45.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.642+0000 7ff771464640 1 Processor -- start 2024-04-26T22:27:45.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.642+0000 7ff771464640 1 -- start start 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff76c15d320 0x7ff76c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff76c0d1090 0x7ff76c0d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c0cf990 0x7ff76c0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff76c0ceeb0 con 0x7ff76c15d320 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff76c15d700 con 0x7ff76c0d1090 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff771464640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c0ce8f0 con 0x7ff76c0cf990 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff76c15d320 0x7ff76c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff76c15d320 0x7ff76c0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43068/0 (socket says 172.21.15.62:43068) 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 -- 172.21.15.62:0/3106735661 learned_addr learned my addr 172.21.15.62:0/3106735661 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76a7fc640 1 --2- 172.21.15.62:0/3106735661 >> v2:172.21.15.174:3300/0 conn(0x7ff76c0d1090 0x7ff76c0d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76b7fe640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c0cf990 0x7ff76c0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 -- 172.21.15.62:0/3106735661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c0cf990 msgr2=0x7ff76c0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c0cf990 0x7ff76c0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 -- 172.21.15.62:0/3106735661 >> v2:172.21.15.174:3300/0 conn(0x7ff76c0d1090 msgr2=0x7ff76c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.643+0000 7ff76affd640 1 --2- 172.21.15.62:0/3106735661 >> v2:172.21.15.174:3300/0 conn(0x7ff76c0d1090 0x7ff76c0d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.644+0000 7ff76affd640 1 -- 172.21.15.62:0/3106735661 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c0cebd0 con 0x7ff76c15d320 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.644+0000 7ff76affd640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c15d320 0x7ff76c0d0b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff764009e50 tx=0x7ff76402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de41e39ff691d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.644+0000 7ff769ffb640 1 -- 172.21.15.62:0/3106735661 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764036070 con 0x7ff76c15d320 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.644+0000 7ff769ffb640 1 -- 172.21.15.62:0/3106735661 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff764031600 con 0x7ff76c15d320 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.644+0000 7ff769ffb640 1 -- 172.21.15.62:0/3106735661 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7640319d0 con 0x7ff76c15d320 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 -- 172.21.15.62:0/3106735661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c15d320 msgr2=0x7ff76c0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c15d320 0x7ff76c0d0b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff764009e50 tx=0x7ff76402a630 comp rx=0 tx=0).stop 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 -- 172.21.15.62:0/3106735661 shutdown_connections 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c0cf990 0x7ff76c0cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 --2- 172.21.15.62:0/3106735661 >> v2:172.21.15.174:3300/0 conn(0x7ff76c0d1090 0x7ff76c0d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 --2- 172.21.15.62:0/3106735661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c15d320 0x7ff76c0d0b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 -- 172.21.15.62:0/3106735661 >> 172.21.15.62:0/3106735661 conn(0x7ff76c0d2f80 msgr2=0x7ff76c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:45.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 -- 172.21.15.62:0/3106735661 shutdown_connections 2024-04-26T22:27:45.648 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.645+0000 7ff771464640 1 -- 172.21.15.62:0/3106735661 wait complete. 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.646+0000 7ff771464640 1 Processor -- start 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.646+0000 7ff771464640 1 -- start start 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0cf990 0x7ff76c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c15d320 0x7ff76c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c0ce550 con 0x7ff76c0d1090 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c0b2cd0 con 0x7ff76c0cf990 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff771464640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff76c0ce6d0 con 0x7ff76c15d320 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff76a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff76a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43082/0 (socket says 172.21.15.62:43082) 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff76a7fc640 1 -- 172.21.15.62:0/3898686279 learned_addr learned my addr 172.21.15.62:0/3898686279 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff76affd640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0cf990 0x7ff76c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.647+0000 7ff76b7fe640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c15d320 0x7ff76c0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 -- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c15d320 msgr2=0x7ff76c0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c15d320 0x7ff76c0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 -- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0cf990 msgr2=0x7ff76c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0cf990 0x7ff76c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 -- 172.21.15.62:0/3898686279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c0cbc90 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff76a7fc640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff75400d5f0 tx=0x7ff75400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540125f0 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff754019070 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c0cc220 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540129d0 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.648+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c0cb880 con 0x7ff76c0d1090 2024-04-26T22:27:45.651 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.649+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76c0693e0 con 0x7ff76c0d1090 2024-04-26T22:27:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.649+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff754012b70 con 0x7ff76c0d1090 2024-04-26T22:27:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.650+0000 7ff7637fe640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.650+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff75408bd30 con 0x7ff76c0d1090 2024-04-26T22:27:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.650+0000 7ff76affd640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.651+0000 7ff76affd640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 0x7ff74c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff764033000 tx=0x7ff764009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.654+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff754059ba0 con 0x7ff76c0d1090 2024-04-26T22:27:45.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.838+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff76c15d880 con 0x7ff76c0d1090 2024-04-26T22:27:45.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.839+0000 7ff7637fe640 1 -- 172.21.15.62:0/3898686279 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7ff75405dea0 con 0x7ff76c0d1090 2024-04-26T22:27:45.840 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.840+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 msgr2=0x7ff74c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.840+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 0x7ff74c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff764033000 tx=0x7ff764009000 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.840+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 msgr2=0x7ff76c0c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.840+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff75400d5f0 tx=0x7ff75400dab0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 shutdown_connections 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff74c06c8f0 0x7ff74c06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff76c15d320 0x7ff76c0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff76c0d1090 0x7ff76c0c91d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 --2- 172.21.15.62:0/3898686279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff76c0cf990 0x7ff76c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 >> 172.21.15.62:0/3898686279 conn(0x7ff76c0d2f80 msgr2=0x7ff76c0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 shutdown_connections 2024-04-26T22:27:45.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.841+0000 7ff771464640 1 -- 172.21.15.62:0/3898686279 wait complete. 2024-04-26T22:27:45.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:45.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 Processor -- start 2024-04-26T22:27:45.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 -- start start 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ea015f540 0x7f1ea006fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea006cb60 con 0x7f1ea0160590 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea0160100 con 0x7f1ea015f540 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.958+0000 7f1ea5672640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea015fad0 con 0x7f1ea006c240 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1ea015f540 0x7f1ea006fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9f7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f1ea0160590 0x7f1ea0073080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60830/0 (socket says 172.21.15.62:60830) 2024-04-26T22:27:45.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9f7fe640 1 -- 172.21.15.62:0/1907937740 learned_addr learned my addr 172.21.15.62:0/1907937740 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/1907937740 >> v2:172.21.15.62:3300/0 conn(0x7f1ea015f540 0x7f1ea006fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43094/0 (socket says 172.21.15.62:43094) 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9effd640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/1907937740 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 msgr2=0x7f1ea006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/1907937740 >> v1:172.21.15.62:6789/0 conn(0x7f1ea0160590 legacy=0x7f1ea0073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9dffb640 1 -- 172.21.15.62:0/1907937740 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318660505 0 0) 0x7f1ea006cb60 con 0x7f1ea0160590 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/1907937740 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea006f890 con 0x7f1ea015f540 2024-04-26T22:27:45.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.959+0000 7f1e9effd640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:45.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea006fdb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097a0 tx=0x7f1e8802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8f07e6f06431db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1e9dffb640 1 -- 172.21.15.62:0/1907937740 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88037070 con 0x7f1ea015f540 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1e9dffb640 1 -- 172.21.15.62:0/1907937740 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8802adb0 con 0x7f1ea015f540 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1e9dffb640 1 -- 172.21.15.62:0/1907937740 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88032660 con 0x7f1ea015f540 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 -- 172.21.15.62:0/1907937740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 msgr2=0x7f1ea006fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea006fdb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097a0 tx=0x7f1e8802a060 comp rx=0 tx=0).stop 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 -- 172.21.15.62:0/1907937740 shutdown_connections 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea006fdb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 --2- 172.21.15.62:0/1907937740 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 -- 172.21.15.62:0/1907937740 >> 172.21.15.62:0/1907937740 conn(0x7f1ea00b3b70 msgr2=0x7f1ea00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 -- 172.21.15.62:0/1907937740 shutdown_connections 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.960+0000 7f1ea5672640 1 -- 172.21.15.62:0/1907937740 wait complete. 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 Processor -- start 2024-04-26T22:27:45.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 -- start start 2024-04-26T22:27:45.965 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea0153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 0x7f1ea01e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea006f2d0 con 0x7f1ea015f540 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea00702f0 con 0x7f1ea0160590 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.962+0000 7f1ea5672640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea006f450 con 0x7f1ea006c240 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43108/0 (socket says 172.21.15.62:43108) 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/3111929285 learned_addr learned my addr 172.21.15.62:0/3111929285 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9effd640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea0153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9f7fe640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 0x7f1ea01e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 msgr2=0x7f1ea0153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea0153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 msgr2=0x7f1ea01e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 0x7f1ea01e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 -- 172.21.15.62:0/3111929285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00b1960 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9f7fe640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 0x7f1ea01e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e9e7fc640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097a0 tx=0x7f1e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88037040 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e88004030 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88004400 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea00b1c40 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.963+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea015c8a0 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.964+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea00693e0 con 0x7f1ea015f540 2024-04-26T22:27:45.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.965+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e8803d650 con 0x7f1ea015f540 2024-04-26T22:27:45.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.965+0000 7f1e977fe640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 0x7f1e8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:45.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.965+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e880ac410 con 0x7f1ea015f540 2024-04-26T22:27:45.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.965+0000 7f1e9effd640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 0x7f1e8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:45.968 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.966+0000 7f1e9effd640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 0x7f1e8006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006fd0 tx=0x7f1e98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:45.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:45.969+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e88042020 con 0x7f1ea015f540 2024-04-26T22:27:46.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.155+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ea00b38e0 con 0x7f1ea015f540 2024-04-26T22:27:46.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.156+0000 7f1e977fe640 1 -- 172.21.15.62:0/3111929285 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f1e8807a840 con 0x7f1ea015f540 2024-04-26T22:27:46.157 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 msgr2=0x7f1e8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 0x7f1e8006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e98006fd0 tx=0x7f1e98008040 comp rx=0 tx=0).stop 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 msgr2=0x7f1ea015d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1e880097a0 tx=0x7f1e880046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 shutdown_connections 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1e8006c8f0 0x7f1e8006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ea0160590 0x7f1ea01e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1ea015f540 0x7f1ea015d0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 --2- 172.21.15.62:0/3111929285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ea006c240 0x7f1ea0153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 >> 172.21.15.62:0/3111929285 conn(0x7f1ea00b3b70 msgr2=0x7f1ea00aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:46.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 shutdown_connections 2024-04-26T22:27:46.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.158+0000 7f1ea5672640 1 -- 172.21.15.62:0/3111929285 wait complete. 2024-04-26T22:27:46.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.272+0000 7f150fcc4640 1 Processor -- start 2024-04-26T22:27:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.272+0000 7f150fcc4640 1 -- start start 2024-04-26T22:27:46.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150fcc4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f150806df90 0x7f15080cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150fcc4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150815c830 con 0x7f150815c450 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150fcc4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f150815d600 con 0x7f15080729f0 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150fcc4640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f150815d8e0 con 0x7f150806df90 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150d25f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f150806df90 0x7f15080cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150da60640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f150815c450 0x7f15080cbcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60844/0 (socket says 172.21.15.62:60844) 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150da60640 1 -- 172.21.15.62:0/2051487030 learned_addr learned my addr 172.21.15.62:0/2051487030 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150d25f640 1 -- 172.21.15.62:0/2051487030 >> v1:172.21.15.174:6789/0 conn(0x7f15080729f0 legacy=0x7f1508072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150d25f640 1 -- 172.21.15.62:0/2051487030 >> v1:172.21.15.62:6789/0 conn(0x7f150815c450 legacy=0x7f15080cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150d25f640 1 -- 172.21.15.62:0/2051487030 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150806f830 con 0x7f150806df90 2024-04-26T22:27:46.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.273+0000 7f150ca5e640 1 -- 172.21.15.62:0/2051487030 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106891670 0 0) 0x7f150815c830 con 0x7f150815c450 2024-04-26T22:27:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.274+0000 7f150d25f640 1 --2- 172.21.15.62:0/2051487030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150806df90 0x7f15080cc3e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd04058c5203e2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.274+0000 7f150ca5e640 1 -- 172.21.15.62:0/2051487030 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8037070 con 0x7f150806df90 2024-04-26T22:27:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.274+0000 7f150ca5e640 1 -- 172.21.15.62:0/2051487030 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f802ae50 con 0x7f150806df90 2024-04-26T22:27:46.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.274+0000 7f150ca5e640 1 -- 172.21.15.62:0/2051487030 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8032660 con 0x7f150806df90 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 -- 172.21.15.62:0/2051487030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150806df90 msgr2=0x7f15080cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 --2- 172.21.15.62:0/2051487030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150806df90 0x7f15080cc3e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097a0 tx=0x7f14f802a140 comp rx=0 tx=0).stop 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 -- 172.21.15.62:0/2051487030 shutdown_connections 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 --2- 172.21.15.62:0/2051487030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150806df90 0x7f15080cc3e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 -- 172.21.15.62:0/2051487030 >> 172.21.15.62:0/2051487030 conn(0x7f15080d0d70 msgr2=0x7f15080d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 -- 172.21.15.62:0/2051487030 shutdown_connections 2024-04-26T22:27:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.275+0000 7f150fcc4640 1 -- 172.21.15.62:0/2051487030 wait complete. 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.276+0000 7f150fcc4640 1 Processor -- start 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.276+0000 7f150fcc4640 1 -- start start 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f150806df90 0x7f15080ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15080729f0 0x7f15080c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f15080ccaa0 con 0x7f150815c450 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15080cc920 con 0x7f15080729f0 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f15080734e0 con 0x7f150806df90 2024-04-26T22:27:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43126/0 (socket says 172.21.15.62:43126) 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 -- 172.21.15.62:0/4231451719 learned_addr learned my addr 172.21.15.62:0/4231451719 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 -- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f150806df90 msgr2=0x7f15080ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f150806df90 0x7f15080ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150d25f640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15080729f0 0x7f15080c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 -- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15080729f0 msgr2=0x7f15080c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15080729f0 0x7f15080c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 -- 172.21.15.62:0/4231451719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150806f3d0 con 0x7f150815c450 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150e261640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f150000c780 tx=0x7f150000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500004b60 con 0x7f150815c450 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500004d00 con 0x7f150815c450 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150809baf0 con 0x7f150815c450 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150000aa90 con 0x7f150815c450 2024-04-26T22:27:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.277+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150809c0b0 con 0x7f150815c450 2024-04-26T22:27:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.279+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150806a360 con 0x7f150815c450 2024-04-26T22:27:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.279+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f150000ac30 con 0x7f150815c450 2024-04-26T22:27:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.279+0000 7f14fe7fc640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 0x7f14ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.279+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f150008c050 con 0x7f150815c450 2024-04-26T22:27:46.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.280+0000 7f150da60640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 0x7f14ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.280+0000 7f150da60640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 0x7f14ec06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1504007fd0 tx=0x7f1504007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.284+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f150005a510 con 0x7f150815c450 2024-04-26T22:27:46.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.467+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f15080b6cf0 con 0x7f150815c450 2024-04-26T22:27:46.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.467+0000 7f14fe7fc640 1 -- 172.21.15.62:0/4231451719 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1500059e10 con 0x7f150815c450 2024-04-26T22:27:46.469 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 msgr2=0x7f14ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 0x7f14ec06edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1504007fd0 tx=0x7f1504007960 comp rx=0 tx=0).stop 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 msgr2=0x7f15080c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f150000c780 tx=0x7f150000cc40 comp rx=0 tx=0).stop 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 shutdown_connections 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f14ec06c8f0 0x7f14ec06edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f150815c450 0x7f15080c8860 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15080729f0 0x7f15080c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 --2- 172.21.15.62:0/4231451719 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f150806df90 0x7f15080ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 >> 172.21.15.62:0/4231451719 conn(0x7f15080d0d70 msgr2=0x7f150815ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:46.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 shutdown_connections 2024-04-26T22:27:46.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.469+0000 7f150fcc4640 1 -- 172.21.15.62:0/4231451719 wait complete. 2024-04-26T22:27:46.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:27:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.583+0000 7f5dcab4d640 1 Processor -- start 2024-04-26T22:27:46.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.583+0000 7f5dcab4d640 1 -- start start 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dcab4d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5dc40d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dcab4d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5dc40b9a90 0x7f5dc40d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dcab4d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc40d3de0 con 0x7f5dc415f760 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dcab4d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc40d43a0 con 0x7f5dc40b9a90 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dcab4d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40d40c0 con 0x7f5dc415f380 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5dc415f760 0x7f5dc40cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60860/0 (socket says 172.21.15.62:60860) 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc3fff640 1 -- 172.21.15.62:0/1136174844 learned_addr learned my addr 172.21.15.62:0/1136174844 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5dc40d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc90ea640 1 --2- 172.21.15.62:0/1136174844 >> v2:172.21.15.174:3300/0 conn(0x7f5dc40b9a90 0x7f5dc40d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955413355 0 0) 0x7f5dc40d3de0 con 0x7f5dc415f760 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5db0003520 con 0x7f5dc415f760 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.584+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2690731765 0 0) 0x7f5db0003520 con 0x7f5dc415f760 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 msgr2=0x7f5dc40d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5dc40d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 >> v2:172.21.15.174:3300/0 conn(0x7f5dc40b9a90 msgr2=0x7f5dc40d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 --2- 172.21.15.62:0/1136174844 >> v2:172.21.15.174:3300/0 conn(0x7f5dc40b9a90 0x7f5dc40d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40b8280 con 0x7f5dc415f760 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f5db40019f0 con 0x7f5dc415f760 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 >> v1:172.21.15.62:6789/0 conn(0x7f5dc415f760 legacy=0x7f5dc40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5db0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d40 0x7f5db00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40d43a0 con 0x7f5db00039b0 2024-04-26T22:27:46.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40d40c0 con 0x7f5db0004d40 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5db000b9c0 con 0x7f5dc415f380 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc90ea640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d40 0x7f5db00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc3fff640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5db0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 -- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 msgr2=0x7f5db0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5db0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 -- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d40 msgr2=0x7f5db00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5db0004d40 0x7f5db00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc88e9640 1 -- 172.21.15.62:0/1136174844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db0003520 con 0x7f5db00039b0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.585+0000 7f5dc3fff640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc415f380 0x7f5db0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5db0003dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009ac0 tx=0x7f5db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94823a571181c606 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc036070 con 0x7f5db00039b0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc031400 con 0x7f5db00039b0 2024-04-26T22:27:46.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dc37fe640 1 -- 172.21.15.62:0/1136174844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc009690 con 0x7f5db00039b0 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dcab4d640 1 -- 172.21.15.62:0/1136174844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 msgr2=0x7f5db0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/1136174844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5db0003dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009ac0 tx=0x7f5db0005830 comp rx=0 tx=0).stop 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dc90ea640 1 -- 172.21.15.62:0/1136174844 reap_dead start 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dcab4d640 1 -- 172.21.15.62:0/1136174844 shutdown_connections 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.586+0000 7f5dcab4d640 1 -- 172.21.15.62:0/1136174844 >> 172.21.15.62:0/1136174844 conn(0x7f5dc409e560 msgr2=0x7f5dc40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.587+0000 7f5dcab4d640 1 -- 172.21.15.62:0/1136174844 shutdown_connections 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.587+0000 7f5dcab4d640 1 -- 172.21.15.62:0/1136174844 wait complete. 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.588+0000 7f5dcab4d640 1 Processor -- start 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.588+0000 7f5dcab4d640 1 -- start start 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cd940 0x7f5dc40cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc40d00d0 0x7f5dc41520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40d1280 con 0x7f5db00039b0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc40d1100 con 0x7f5dc40d00d0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40d5b50 con 0x7f5dc40cd940 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43146/0 (socket says 172.21.15.62:43146) 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 -- 172.21.15.62:0/2775590417 learned_addr learned my addr 172.21.15.62:0/2775590417 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc90ea640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc40d00d0 0x7f5dc41520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc3fff640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cd940 0x7f5dc40cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 -- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cd940 msgr2=0x7f5dc40cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cd940 0x7f5dc40cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 -- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc40d00d0 msgr2=0x7f5dc41520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc40d00d0 0x7f5dc41520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 -- 172.21.15.62:0/2775590417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc40cde70 con 0x7f5db00039b0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc88e9640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc034040 tx=0x7f5dbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.594 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc040020 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc003bc0 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc40ce150 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc036040 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.589+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc40ce710 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.590+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc406a360 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.591+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dbc043020 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.591+0000 7f5dc17fa640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.591+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5dbc0abae0 con 0x7f5db00039b0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.592+0000 7f5dc3fff640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:46.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.592+0000 7f5dc3fff640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c8770 tx=0x7f5db4001890 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:46.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.596+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5dbc04a070 con 0x7f5db00039b0 2024-04-26T22:27:46.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.792+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5dc40ba0f0 con 0x7f5db00039b0 2024-04-26T22:27:46.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.793+0000 7f5dc17fa640 1 -- 172.21.15.62:0/2775590417 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f5dbc0044e0 con 0x7f5db00039b0 2024-04-26T22:27:46.794 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:27:46.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 msgr2=0x7f5da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 0x7f5da406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5dc40c8770 tx=0x7f5db4001890 comp rx=0 tx=0).stop 2024-04-26T22:27:46.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 msgr2=0x7f5dc40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:46.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc034040 tx=0x7f5dbc004730 comp rx=0 tx=0).stop 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 shutdown_connections 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5da406c8f0 0x7f5da406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc40d00d0 0x7f5dc41520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dc40cd940 0x7f5dc40cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 --2- 172.21.15.62:0/2775590417 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5db00039b0 0x7f5dc40cef10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 >> 172.21.15.62:0/2775590417 conn(0x7f5dc409e560 msgr2=0x7f5dc406d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 shutdown_connections 2024-04-26T22:27:46.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:46.795+0000 7f5dcab4d640 1 -- 172.21.15.62:0/2775590417 wait complete. 2024-04-26T22:27:46.807 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T22:27:46.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:46.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T22:27:46.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:27:46.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:46.833 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T22:27:46.888 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:46.888 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:46.907 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:46.907 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:46.965 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:27:46.966 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:27:47.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:47.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.123+0000 7f12ea7bf640 1 Processor -- start 2024-04-26T22:27:47.124 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.123+0000 7f12ea7bf640 1 -- start start 2024-04-26T22:27:47.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12ea7bf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12e40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12ea7bf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e40b1d70 con 0x7f12e4162e40 2024-04-26T22:27:47.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12ea7bf640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e40d1960 con 0x7f12e40cf830 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12ea7bf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f12e4163220 con 0x7f12e406ea40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f12e4162e40 0x7f12e40b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60874/0 (socket says 172.21.15.62:60874) 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e3fff640 1 -- 172.21.15.62:0/2233606742 learned_addr learned my addr 172.21.15.62:0/2233606742 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e37fe640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12e40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1142831617 0 0) 0x7f12e40b1d70 con 0x7f12e4162e40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12cc003520 con 0x7f12e4162e40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 341931605 0 0) 0x7f12cc003520 con 0x7f12e4162e40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 msgr2=0x7f12e40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12e40d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 >> v1:172.21.15.174:6789/0 conn(0x7f12e40cf830 legacy=0x7f12e40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e40d1c40 con 0x7f12e4162e40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f12d8001670 con 0x7f12e4162e40 2024-04-26T22:27:47.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.124+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 >> v1:172.21.15.62:6789/0 conn(0x7f12e4162e40 legacy=0x7f12e40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc0039e0 0x7f12cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12cc004390 0x7f12cc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12cc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc00ba70 con 0x7f12cc004390 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc00bc20 con 0x7f12cc0039e0 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc00be20 con 0x7f12e406ea40 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12cc004390 0x7f12cc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e3fff640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc0039e0 0x7f12cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e8d5c640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12cc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 -- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 msgr2=0x7f12cc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12cc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 -- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc0039e0 msgr2=0x7f12cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:47.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc0039e0 0x7f12cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e37fe640 1 -- 172.21.15.62:0/2233606742 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc003520 con 0x7f12cc004390 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e3fff640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc0039e0 0x7f12cc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.125+0000 7f12e8d5c640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e406ea40 0x7f12cc005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12e37fe640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12cc004390 0x7f12cc004860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009aa0 tx=0x7f12cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3339afd3568a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4036070 con 0x7f12cc004390 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d4031400 con 0x7f12cc004390 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12e2ffd640 1 -- 172.21.15.62:0/2233606742 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4009670 con 0x7f12cc004390 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 -- 172.21.15.62:0/2233606742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12cc004390 msgr2=0x7f12cc004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/2233606742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12cc004390 0x7f12cc004860 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009aa0 tx=0x7f12cc005830 comp rx=0 tx=0).stop 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12e8d5c640 1 -- 172.21.15.62:0/2233606742 reap_dead start 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 -- 172.21.15.62:0/2233606742 shutdown_connections 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 -- 172.21.15.62:0/2233606742 >> 172.21.15.62:0/2233606742 conn(0x7f12e409ba50 msgr2=0x7f12e409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 -- 172.21.15.62:0/2233606742 shutdown_connections 2024-04-26T22:27:47.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.126+0000 7f12ea7bf640 1 -- 172.21.15.62:0/2233606742 wait complete. 2024-04-26T22:27:47.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.127+0000 7f12ea7bf640 1 Processor -- start 2024-04-26T22:27:47.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 -- start start 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc004390 0x7f12e4155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e40cb800 0x7f12e4160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40d1340 con 0x7f12e415e8b0 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12e40d0eb0 con 0x7f12cc004390 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12ea7bf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f12e40d14c0 con 0x7f12e40cb800 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.128+0000 7f12e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43178/0 (socket says 172.21.15.62:43178) 2024-04-26T22:27:47.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 -- 172.21.15.62:0/885357102 learned_addr learned my addr 172.21.15.62:0/885357102 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e3fff640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc004390 0x7f12e4155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e8d5c640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e40cb800 0x7f12e4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 -- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e40cb800 msgr2=0x7f12e4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e40cb800 0x7f12e4160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 -- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc004390 msgr2=0x7f12e4155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc004390 0x7f12e4155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 -- 172.21.15.62:0/885357102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e415ed30 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e37fe640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12d402ab70 tx=0x7f12d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4036040 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d403d680 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4009670 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e415d810 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.129+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e415ddd0 con 0x7f12e415e8b0 2024-04-26T22:27:47.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.130+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e40ce510 con 0x7f12e415e8b0 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.131+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12d4039040 con 0x7f12e415e8b0 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.131+0000 7f12e0ff9640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 0x7f12d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.131+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f12d40ac350 con 0x7f12e415e8b0 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.131+0000 7f12e3fff640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 0x7f12d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.132+0000 7f12e3fff640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 0x7f12d006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005a30 tx=0x7f12d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.136+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f12d404a030 con 0x7f12e415e8b0 2024-04-26T22:27:47.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.321+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12e406a360 con 0x7f12e415e8b0 2024-04-26T22:27:47.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.322+0000 7f12e0ff9640 1 -- 172.21.15.62:0/885357102 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f12d4041020 con 0x7f12e415e8b0 2024-04-26T22:27:47.324 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:47.324 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 msgr2=0x7f12d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 0x7f12d006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005a30 tx=0x7f12d8009040 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 msgr2=0x7f12e40cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12d402ab70 tx=0x7f12d4004730 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 shutdown_connections 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f12d006c8f0 0x7f12d006edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12e40cb800 0x7f12e4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f12e415e8b0 0x7f12e40cb2c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 --2- 172.21.15.62:0/885357102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12cc004390 0x7f12e4155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.326 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.324+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 >> 172.21.15.62:0/885357102 conn(0x7f12e409ba50 msgr2=0x7f12e40b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.325+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 shutdown_connections 2024-04-26T22:27:47.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.325+0000 7f12ea7bf640 1 -- 172.21.15.62:0/885357102 wait complete. 2024-04-26T22:27:47.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:47.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.440+0000 7f3220009640 1 Processor -- start 2024-04-26T22:27:47.441 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.440+0000 7f3220009640 1 -- start start 2024-04-26T22:27:47.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f3220009640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f321815d460 0x7f321806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f3220009640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3218073c40 0x7f32180ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f3220009640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f321806ece0 con 0x7f32180cd100 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f3220009640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f32180add80 con 0x7f321815d460 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f3220009640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f321806efc0 con 0x7f3218073c40 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f321815d460 0x7f321806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f321815d460 0x7f321806e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43192/0 (socket says 172.21.15.62:43192) 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 -- 172.21.15.62:0/382309618 learned_addr learned my addr 172.21.15.62:0/382309618 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321d5a4640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3218073c40 0x7f32180ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 -- 172.21.15.62:0/382309618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3218073c40 msgr2=0x7f32180ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3218073c40 0x7f32180ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 -- 172.21.15.62:0/382309618 >> v1:172.21.15.174:6789/0 conn(0x7f32180cd100 legacy=0x7f32180cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.441+0000 7f321dda5640 1 -- 172.21.15.62:0/382309618 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321806f2a0 con 0x7f321815d460 2024-04-26T22:27:47.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f321dda5640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321806e700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3214009870 tx=0x7f321402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768ccdfe521e88ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f321cda3640 1 -- 172.21.15.62:0/382309618 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214037070 con 0x7f321815d460 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f321cda3640 1 -- 172.21.15.62:0/382309618 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321402ae50 con 0x7f321815d460 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f321cda3640 1 -- 172.21.15.62:0/382309618 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214032860 con 0x7f321815d460 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f3220009640 1 -- 172.21.15.62:0/382309618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 msgr2=0x7f321806e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.442+0000 7f3220009640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321806e700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3214009870 tx=0x7f321402a300 comp rx=0 tx=0).stop 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 -- 172.21.15.62:0/382309618 shutdown_connections 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3218073c40 0x7f32180ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 --2- 172.21.15.62:0/382309618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321806e700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 -- 172.21.15.62:0/382309618 >> 172.21.15.62:0/382309618 conn(0x7f32180d0d70 msgr2=0x7f32180d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 -- 172.21.15.62:0/382309618 shutdown_connections 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.443+0000 7f3220009640 1 -- 172.21.15.62:0/382309618 wait complete. 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.444+0000 7f3220009640 1 Processor -- start 2024-04-26T22:27:47.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.444+0000 7f3220009640 1 -- start start 2024-04-26T22:27:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.444+0000 7f3220009640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218073c40 0x7f32181511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32180cd100 0x7f3218154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f321815d9c0 con 0x7f321815d460 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f321815d840 con 0x7f3218073c40 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f32180740a0 con 0x7f32180cd100 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43198/0 (socket says 172.21.15.62:43198) 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 -- 172.21.15.62:0/3812839030 learned_addr learned my addr 172.21.15.62:0/3812839030 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321dda5640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218073c40 0x7f32181511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321d5a4640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32180cd100 0x7f3218154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 -- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32180cd100 msgr2=0x7f3218154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32180cd100 0x7f3218154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 -- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218073c40 msgr2=0x7f32181511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218073c40 0x7f32181511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 -- 172.21.15.62:0/3812839030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3218154c30 con 0x7f321815d460 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f321e5a6640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f321000c7d0 tx=0x7f321000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210003b70 con 0x7f321815d460 2024-04-26T22:27:47.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3210003d30 con 0x7f321815d460 2024-04-26T22:27:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3218151cf0 con 0x7f321815d460 2024-04-26T22:27:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000aa30 con 0x7f321815d460 2024-04-26T22:27:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.445+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32181522b0 con 0x7f321815d460 2024-04-26T22:27:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.446+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321806a360 con 0x7f321815d460 2024-04-26T22:27:47.449 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.447+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f321000abd0 con 0x7f321815d460 2024-04-26T22:27:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.447+0000 7f320e7fc640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 0x7f31fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.447+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f321008b980 con 0x7f321815d460 2024-04-26T22:27:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.447+0000 7f321dda5640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 0x7f31fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.448+0000 7f321dda5640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 0x7f31fc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3214034000 tx=0x7f321402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.451+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3210059e40 con 0x7f321815d460 2024-04-26T22:27:47.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.638+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f321815ea80 con 0x7f321815d460 2024-04-26T22:27:47.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.639+0000 7f320e7fc640 1 -- 172.21.15.62:0/3812839030 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3210059740 con 0x7f321815d460 2024-04-26T22:27:47.640 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:47.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 msgr2=0x7f31fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 0x7f31fc06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3214034000 tx=0x7f321402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:47.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 msgr2=0x7f321809df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f321000c7d0 tx=0x7f321000cc90 comp rx=0 tx=0).stop 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 shutdown_connections 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f31fc06c8f0 0x7f31fc06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f321815d460 0x7f321809df10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32180cd100 0x7f3218154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 --2- 172.21.15.62:0/3812839030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218073c40 0x7f32181511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 >> 172.21.15.62:0/3812839030 conn(0x7f32180d0d70 msgr2=0x7f321815f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 shutdown_connections 2024-04-26T22:27:47.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.641+0000 7f3220009640 1 -- 172.21.15.62:0/3812839030 wait complete. 2024-04-26T22:27:47.653 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.756+0000 7f4fc2796640 1 Processor -- start 2024-04-26T22:27:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.756+0000 7f4fc2796640 1 -- start start 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc2796640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fbc0d0dd0 0x7f4fbc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc2796640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fbc0d09d0 con 0x7f4fbc15d630 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc2796640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fbc0cec60 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc2796640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f4fbc0cf500 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc0d33640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4fbc0d0dd0 0x7f4fbc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fbb7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4fbc15d630 0x7f4fbc0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60882/0 (socket says 172.21.15.62:60882) 2024-04-26T22:27:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fbb7fe640 1 -- 172.21.15.62:0/3816272916 learned_addr learned my addr 172.21.15.62:0/3816272916 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc0d33640 1 -- 172.21.15.62:0/3816272916 >> v1:172.21.15.174:6789/0 conn(0x7f4fbc0d15d0 legacy=0x7f4fbc0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc0d33640 1 -- 172.21.15.62:0/3816272916 >> v1:172.21.15.62:6789/0 conn(0x7f4fbc15d630 legacy=0x7f4fbc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.757+0000 7f4fc0d33640 1 -- 172.21.15.62:0/3816272916 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc0cef40 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fc0d33640 1 --2- 172.21.15.62:0/3816272916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d0dd0 0x7f4fbc0b31a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009d90 tx=0x7f4fb402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b259187c254b4bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fbaffd640 1 -- 172.21.15.62:0/3816272916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031600 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fbaffd640 1 -- 172.21.15.62:0/3816272916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb40317a0 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fbaffd640 1 -- 172.21.15.62:0/3816272916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031b70 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fc2796640 1 -- 172.21.15.62:0/3816272916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d0dd0 msgr2=0x7f4fbc0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.758+0000 7f4fc2796640 1 --2- 172.21.15.62:0/3816272916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d0dd0 0x7f4fbc0b31a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009d90 tx=0x7f4fb402a610 comp rx=0 tx=0).stop 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.759+0000 7f4fc2796640 1 -- 172.21.15.62:0/3816272916 shutdown_connections 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.759+0000 7f4fc2796640 1 --2- 172.21.15.62:0/3816272916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d0dd0 0x7f4fbc0b31a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.759+0000 7f4fc2796640 1 -- 172.21.15.62:0/3816272916 >> 172.21.15.62:0/3816272916 conn(0x7f4fbc0d2ff0 msgr2=0x7f4fbc0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.759+0000 7f4fc2796640 1 -- 172.21.15.62:0/3816272916 shutdown_connections 2024-04-26T22:27:47.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.759+0000 7f4fc2796640 1 -- 172.21.15.62:0/3816272916 wait complete. 2024-04-26T22:27:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.760+0000 7f4fc2796640 1 Processor -- start 2024-04-26T22:27:47.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.760+0000 7f4fc2796640 1 -- start start 2024-04-26T22:27:47.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 0x7f4fbc154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 0x7f4fbc0c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc0ce8f0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc0b2570 con 0x7f4fbc15d630 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc2796640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc0c67a0 con 0x7f4fbc0d0dd0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43206/0 (socket says 172.21.15.62:43206) 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 -- 172.21.15.62:0/453323298 learned_addr learned my addr 172.21.15.62:0/453323298 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc0d33640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 0x7f4fbc0c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbbfff640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 0x7f4fbc154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 -- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 msgr2=0x7f4fbc154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 0x7f4fbc154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 -- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 msgr2=0x7f4fbc0c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 0x7f4fbc0c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 -- 172.21.15.62:0/453323298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc0ca0c0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbbfff640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 0x7f4fbc154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:47.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fc0d33640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 0x7f4fbc0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fbb7fe640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c440 tx=0x7f4fb000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.761+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0017070 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.762+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fb0005ce0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.762+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc0ca3a0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.762+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00072f0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.765 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.762+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc0ca960 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.763+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc0cd6e0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.763+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fb001a070 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.763+0000 7f4fb8ff9640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.763+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4fb008add0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.764+0000 7f4fbbfff640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:47.766 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.764+0000 7f4fbbfff640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 0x7f4fa806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00a830 tx=0x7f4fac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:47.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.768+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fbc0cd6e0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.952+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4fbc0744b0 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.952+0000 7f4fb8ff9640 1 -- 172.21.15.62:0/453323298 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f4fb0059290 con 0x7f4fbc0d15d0 2024-04-26T22:27:47.954 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 msgr2=0x7f4fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 0x7f4fa806edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00a830 tx=0x7f4fac008040 comp rx=0 tx=0).stop 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 msgr2=0x7f4fbc0b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c440 tx=0x7f4fb000c900 comp rx=0 tx=0).stop 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 shutdown_connections 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4fa806c8f0 0x7f4fa806edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fbc15d630 0x7f4fbc0c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4fbc0d15d0 0x7f4fbc0b6850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.954+0000 7f4fc2796640 1 --2- 172.21.15.62:0/453323298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4fbc0d0dd0 0x7f4fbc154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:47.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.955+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 >> 172.21.15.62:0/453323298 conn(0x7f4fbc0d2ff0 msgr2=0x7f4fbc1563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:47.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.955+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 shutdown_connections 2024-04-26T22:27:47.957 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:47.955+0000 7f4fc2796640 1 -- 172.21.15.62:0/453323298 wait complete. 2024-04-26T22:27:47.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.069+0000 7f661e0c8640 1 Processor -- start 2024-04-26T22:27:48.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.069+0000 7f661e0c8640 1 -- start start 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f661e0c8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f661806e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f661e0c8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6618073c40 0x7f66180ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f661e0c8640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f661806ece0 con 0x7f66180cd100 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f661e0c8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f66180add80 con 0x7f6618073c40 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f661e0c8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f661806efc0 con 0x7f661815d460 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f6617fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f66180cd100 0x7f66180cb3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60898/0 (socket says 172.21.15.62:60898) 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f6617fff640 1 -- 172.21.15.62:0/3745739406 learned_addr learned my addr 172.21.15.62:0/3745739406 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:48.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f6616ffd640 1 --2- 172.21.15.62:0/3745739406 >> v2:172.21.15.174:3300/0 conn(0x7f6618073c40 0x7f66180ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f66177fe640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f661806e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1319555286 0 0) 0x7f661806ece0 con 0x7f66180cd100 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.070+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6600003520 con 0x7f66180cd100 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1873161397 0 0) 0x7f6600003520 con 0x7f66180cd100 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 msgr2=0x7f661806e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f661806e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 >> v2:172.21.15.174:3300/0 conn(0x7f6618073c40 msgr2=0x7f66180ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 --2- 172.21.15.62:0/3745739406 >> v2:172.21.15.174:3300/0 conn(0x7f6618073c40 0x7f66180ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661806f2a0 con 0x7f66180cd100 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f66100019f0 con 0x7f66180cd100 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 >> v1:172.21.15.62:6789/0 conn(0x7f66180cd100 legacy=0x7f66180cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f66000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6600003ef0 0x7f6600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6600004d40 0x7f66000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66180add80 con 0x7f6600004d40 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f661806efc0 con 0x7f6600003ef0 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f660000b9c0 con 0x7f661815d460 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6600004d40 0x7f66000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6616ffd640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6600003ef0 0x7f6600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f66177fe640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f66000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 -- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 msgr2=0x7f66000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661815d460 0x7f66000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 -- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6600003ef0 msgr2=0x7f6600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6600003ef0 0x7f6600004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6617fff640 1 -- 172.21.15.62:0/3745739406 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6600003520 con 0x7f6600004d40 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.071+0000 7f6616ffd640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6600003ef0 0x7f6600004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.072+0000 7f6617fff640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6600004d40 0x7f66000051c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6610005b20 tx=0x7f6600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1d413aaed71e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.072+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66100056f0 con 0x7f6600004d40 2024-04-26T22:27:48.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.072+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66100082d0 con 0x7f6600004d40 2024-04-26T22:27:48.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.072+0000 7f66167fc640 1 -- 172.21.15.62:0/3745739406 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610008700 con 0x7f6600004d40 2024-04-26T22:27:48.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 -- 172.21.15.62:0/3745739406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6600004d40 msgr2=0x7f66000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 --2- 172.21.15.62:0/3745739406 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6600004d40 0x7f66000051c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6610005b20 tx=0x7f6600005830 comp rx=0 tx=0).stop 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f6617fff640 1 -- 172.21.15.62:0/3745739406 reap_dead start 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 -- 172.21.15.62:0/3745739406 shutdown_connections 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 -- 172.21.15.62:0/3745739406 >> 172.21.15.62:0/3745739406 conn(0x7f66180d0d70 msgr2=0x7f66180d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 -- 172.21.15.62:0/3745739406 shutdown_connections 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.073+0000 7f661e0c8640 1 -- 172.21.15.62:0/3745739406 wait complete. 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.074+0000 7f661e0c8640 1 Processor -- start 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.074+0000 7f661e0c8640 1 -- start start 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66180d0020 0x7f66180c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661809dc20 0x7f661809e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f661815d9c0 con 0x7f6618157620 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f661815d840 con 0x7f66180d0020 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f661e0c8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f66180740a0 con 0x7f661809dc20 2024-04-26T22:27:48.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43224/0 (socket says 172.21.15.62:43224) 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 -- 172.21.15.62:0/2519447481 learned_addr learned my addr 172.21.15.62:0/2519447481 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 -- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661809dc20 msgr2=0x7f661809e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661809dc20 0x7f661809e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 -- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66180d0020 msgr2=0x7f66180c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f6616ffd640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66180d0020 0x7f66180c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66180d0020 0x7f66180c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 -- 172.21.15.62:0/2519447481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6618157a00 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f66177fe640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6608011920 tx=0x7f6608011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660800faf0 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.075+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f660800fc90 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.076+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661815e4b0 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.076+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608014440 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.076+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f661815ea70 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.077+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f661809e5e0 con 0x7f6618157620 2024-04-26T22:27:48.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.077+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6608012040 con 0x7f6618157620 2024-04-26T22:27:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.077+0000 7f65fffff640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 0x7f660406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.078+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f660808c080 con 0x7f6618157620 2024-04-26T22:27:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.078+0000 7f6616ffd640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 0x7f660406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.078+0000 7f6616ffd640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 0x7f660406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66180b2840 tx=0x7f660c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.082+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6608019030 con 0x7f6618157620 2024-04-26T22:27:48.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.265+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f661806a360 con 0x7f6618157620 2024-04-26T22:27:48.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.265+0000 7f65fffff640 1 -- 172.21.15.62:0/2519447481 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f6608059ef0 con 0x7f6618157620 2024-04-26T22:27:48.267 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:48.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 msgr2=0x7f660406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 0x7f660406ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66180b2840 tx=0x7f660c008040 comp rx=0 tx=0).stop 2024-04-26T22:27:48.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 msgr2=0x7f661815db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6608011920 tx=0x7f6608011de0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 shutdown_connections 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f660406c920 0x7f660406ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f661809dc20 0x7f661809e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66180d0020 0x7f66180c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 --2- 172.21.15.62:0/2519447481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6618157620 0x7f661815db40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 >> 172.21.15.62:0/2519447481 conn(0x7f66180d0d70 msgr2=0x7f66180cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 shutdown_connections 2024-04-26T22:27:48.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.267+0000 7f661e0c8640 1 -- 172.21.15.62:0/2519447481 wait complete. 2024-04-26T22:27:48.279 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:27:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.380+0000 7f195632b640 1 Processor -- start 2024-04-26T22:27:48.381 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.380+0000 7f195632b640 1 -- start start 2024-04-26T22:27:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f195632b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f195006ea40 0x7f19500d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f195632b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f195632b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19500b1d70 con 0x7f1950162e40 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f195632b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1950163220 con 0x7f195006ea40 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f195632b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f19500d01c0 con 0x7f19500cf830 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f195006ea40 0x7f19500d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f195006ea40 0x7f19500d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43240/0 (socket says 172.21.15.62:43240) 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 -- 172.21.15.62:0/3535999412 learned_addr learned my addr 172.21.15.62:0/3535999412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f19548c8640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 -- 172.21.15.62:0/3535999412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 msgr2=0x7f19500cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 -- 172.21.15.62:0/3535999412 >> v1:172.21.15.174:6789/0 conn(0x7f1950162e40 legacy=0x7f19500b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.381+0000 7f194f7fe640 1 -- 172.21.15.62:0/3535999412 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19500d1960 con 0x7f195006ea40 2024-04-26T22:27:48.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.382+0000 7f194f7fe640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195006ea40 0x7f19500d1340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1940009870 tx=0x7f194002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06c135bad281cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.382+0000 7f194effd640 1 -- 172.21.15.62:0/3535999412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940037070 con 0x7f195006ea40 2024-04-26T22:27:48.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.382+0000 7f194effd640 1 -- 172.21.15.62:0/3535999412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1940032440 con 0x7f195006ea40 2024-04-26T22:27:48.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.382+0000 7f194effd640 1 -- 172.21.15.62:0/3535999412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940032810 con 0x7f195006ea40 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 -- 172.21.15.62:0/3535999412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195006ea40 msgr2=0x7f19500d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195006ea40 0x7f19500d1340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1940009870 tx=0x7f194002a1c0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 -- 172.21.15.62:0/3535999412 shutdown_connections 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 --2- 172.21.15.62:0/3535999412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f195006ea40 0x7f19500d1340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 -- 172.21.15.62:0/3535999412 >> 172.21.15.62:0/3535999412 conn(0x7f195009ba50 msgr2=0x7f195009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 -- 172.21.15.62:0/3535999412 shutdown_connections 2024-04-26T22:27:48.385 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.383+0000 7f195632b640 1 -- 172.21.15.62:0/3535999412 wait complete. 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.384+0000 7f195632b640 1 Processor -- start 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.384+0000 7f195632b640 1 -- start start 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f195006ea40 0x7f19500b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d1f20 con 0x7f1950162e40 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19500d0eb0 con 0x7f195006ea40 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f195632b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f19500ce010 con 0x7f19500cf830 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43254/0 (socket says 172.21.15.62:43254) 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 -- 172.21.15.62:0/3546844578 learned_addr learned my addr 172.21.15.62:0/3546844578 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f194f7fe640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f194ffff640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f195006ea40 0x7f19500b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 -- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 msgr2=0x7f19500b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 -- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f195006ea40 msgr2=0x7f19500b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f195006ea40 0x7f19500b3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.385+0000 7f19548c8640 1 -- 172.21.15.62:0/3546844578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195015e0d0 con 0x7f1950162e40 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f19548c8640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1948009dd0 tx=0x7f194800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948018020 con 0x7f1950162e40 2024-04-26T22:27:48.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1948014480 con 0x7f1950162e40 2024-04-26T22:27:48.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948014870 con 0x7f1950162e40 2024-04-26T22:27:48.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195015e3b0 con 0x7f1950162e40 2024-04-26T22:27:48.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.386+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195015e970 con 0x7f1950162e40 2024-04-26T22:27:48.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.387+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195006a360 con 0x7f1950162e40 2024-04-26T22:27:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.387+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1948005d10 con 0x7f1950162e40 2024-04-26T22:27:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.388+0000 7f194cff9640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.388+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f194808c540 con 0x7f1950162e40 2024-04-26T22:27:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.388+0000 7f194ffff640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.388+0000 7f194ffff640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 0x7f193c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19500d0580 tx=0x7f1944009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.392+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1948022070 con 0x7f1950162e40 2024-04-26T22:27:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.589+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f19500d13d0 con 0x7f1950162e40 2024-04-26T22:27:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.589+0000 7f194cff9640 1 -- 172.21.15.62:0/3546844578 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f1948010040 con 0x7f1950162e40 2024-04-26T22:27:48.590 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:27:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 msgr2=0x7f193c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.592 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 0x7f193c06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f19500d0580 tx=0x7f1944009040 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 msgr2=0x7f19500b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1948009dd0 tx=0x7f194800c490 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 shutdown_connections 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f193c06c8f0 0x7f193c06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1950162e40 0x7f19500b6420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f19500cf830 0x7f19500b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 --2- 172.21.15.62:0/3546844578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f195006ea40 0x7f19500b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 >> 172.21.15.62:0/3546844578 conn(0x7f195009ba50 msgr2=0x7f19500ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 shutdown_connections 2024-04-26T22:27:48.593 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.591+0000 7f195632b640 1 -- 172.21.15.62:0/3546844578 wait complete. 2024-04-26T22:27:48.603 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T22:27:48.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:48.603 DEBUG:teuthology.orchestra.run.smithi174:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T22:27:48.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:27:48.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:27:48.634 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T22:27:48.691 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T22:27:48.691 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.790+0000 7fa5ad8dc640 1 Processor -- start 2024-04-26T22:27:48.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 -- start start 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa5a806fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa5a815f4f0 0x7fa5a806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a806cb60 con 0x7fa5a80b1480 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80b59b0 con 0x7fa5a815f4f0 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5ad8dc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a8070350 con 0x7fa5a806c240 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5a77fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa5a80b1480 0x7fa5a80ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60908/0 (socket says 172.21.15.62:60908) 2024-04-26T22:27:48.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.791+0000 7fa5a77fe640 1 -- 172.21.15.62:0/2742343703 learned_addr learned my addr 172.21.15.62:0/2742343703 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:48.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989653704 0 0) 0x7fa5a806cb60 con 0x7fa5a80b1480 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa594003520 con 0x7fa5a80b1480 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a6ffd640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa5a806fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a67fc640 1 --2- 172.21.15.62:0/2742343703 >> v2:172.21.15.174:3300/0 conn(0x7fa5a815f4f0 0x7fa5a806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4175474386 0 0) 0x7fa594003520 con 0x7fa5a80b1480 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 msgr2=0x7fa5a806fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa5a806fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 >> v2:172.21.15.174:3300/0 conn(0x7fa5a815f4f0 msgr2=0x7fa5a806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 --2- 172.21.15.62:0/2742343703 >> v2:172.21.15.174:3300/0 conn(0x7fa5a815f4f0 0x7fa5a806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a80b5c70 con 0x7fa5a80b1480 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fa5a00019f0 con 0x7fa5a80b1480 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 >> v1:172.21.15.62:6789/0 conn(0x7fa5a80b1480 legacy=0x7fa5a80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5940039b0 0x7fa594003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa594004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80b59b0 con 0x7fa594004d40 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8070350 con 0x7fa5940039b0 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa59400b9c0 con 0x7fa5a806c240 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a77fe640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a6ffd640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5940039b0 0x7fa594003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.792+0000 7fa5a67fc640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa594004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 -- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 msgr2=0x7fa594004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a806c240 0x7fa594004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 -- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5940039b0 msgr2=0x7fa594003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5940039b0 0x7fa594003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 -- 172.21.15.62:0/2742343703 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa594003520 con 0x7fa594004d40 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a77fe640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00013d0 tx=0x7fa594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351093a0525687a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000a4e0 con 0x7fa594004d40 2024-04-26T22:27:48.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.793+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a002f070 con 0x7fa594004d40 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5a5ffb640 1 -- 172.21.15.62:0/2742343703 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a002bce0 con 0x7fa594004d40 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/2742343703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa594004d40 msgr2=0x7fa5940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/2742343703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00013d0 tx=0x7fa594005830 comp rx=0 tx=0).stop 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5a77fe640 1 -- 172.21.15.62:0/2742343703 reap_dead start 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/2742343703 shutdown_connections 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/2742343703 >> 172.21.15.62:0/2742343703 conn(0x7fa5a80b3b70 msgr2=0x7fa5a80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/2742343703 shutdown_connections 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.794+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/2742343703 wait complete. 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.795+0000 7fa5ad8dc640 1 Processor -- start 2024-04-26T22:27:48.799 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.795+0000 7fa5ad8dc640 1 -- start start 2024-04-26T22:27:48.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a81f4890 0x7fa5a815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a8147210 0x7fa5a81476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80b0fc0 con 0x7fa5a8147f50 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80b6230 con 0x7fa5a81f4890 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5ad8dc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a806fd70 con 0x7fa5a8147210 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5a67fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43272/0 (socket says 172.21.15.62:43272) 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5a67fc640 1 -- 172.21.15.62:0/3691056867 learned_addr learned my addr 172.21.15.62:0/3691056867 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5a6ffd640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a81f4890 0x7fa5a815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.796+0000 7fa5a77fe640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a8147210 0x7fa5a81476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 -- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a8147210 msgr2=0x7fa5a81476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a8147210 0x7fa5a81476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 -- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a81f4890 msgr2=0x7fa5a815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a81f4890 0x7fa5a815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 -- 172.21.15.62:0/3691056867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8147be0 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5a67fc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa598011a70 tx=0x7fa598011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800fa90 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59800fc30 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a815d260 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598014440 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.797+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a815d820 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.798+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8069c20 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.798+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa598012040 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.799+0000 7fa59ffff640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 0x7fa58806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.799+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa59808c090 con 0x7fa5a8147f50 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.799+0000 7fa5a6ffd640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 0x7fa58806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:48.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.800+0000 7fa5a6ffd640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 0x7fa58806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa590009b40 tx=0x7fa590009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:48.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:48.803+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa5a8069c20 con 0x7fa5a8147f50 2024-04-26T22:27:49.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.026+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5a806a360 con 0x7fa5a8147f50 2024-04-26T22:27:49.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.026+0000 7fa59ffff640 1 -- 172.21.15.62:0/3691056867 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fa5a8147be0 con 0x7fa5a8147f50 2024-04-26T22:27:49.028 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:49.028 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:49.028 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.028+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 msgr2=0x7fa58806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.028+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 0x7fa58806ede0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa590009b40 tx=0x7fa590009190 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 msgr2=0x7fa5a81f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa598011a70 tx=0x7fa598011e70 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 shutdown_connections 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa58806c920 0x7fa58806ede0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a8147210 0x7fa5a81476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa5a8147f50 0x7fa5a81f4c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 --2- 172.21.15.62:0/3691056867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a81f4890 0x7fa5a815c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 >> 172.21.15.62:0/3691056867 conn(0x7fa5a80b3b70 msgr2=0x7fa5a80b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 shutdown_connections 2024-04-26T22:27:49.030 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.029+0000 7fa5ad8dc640 1 -- 172.21.15.62:0/3691056867 wait complete. 2024-04-26T22:27:49.041 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:49.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.140+0000 7f2ea8540640 1 Processor -- start 2024-04-26T22:27:49.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.140+0000 7f2ea8540640 1 -- start start 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea8540640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2ea00d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea8540640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea00b1d70 con 0x7f2ea00b2050 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea8540640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea00d1960 con 0x7f2ea00cff40 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea8540640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea0163220 con 0x7f2ea0162e40 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea5adb640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2ea00b2050 0x7f2ea00cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60912/0 (socket says 172.21.15.62:60912) 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea5adb640 1 -- 172.21.15.62:0/282483141 learned_addr learned my addr 172.21.15.62:0/282483141 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3105153163 0 0) 0x7f2ea00b1d70 con 0x7f2ea00b2050 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e88003520 con 0x7f2ea00b2050 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.141+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2ea00d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1580644530 0 0) 0x7f2e88003520 con 0x7f2ea00b2050 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 msgr2=0x7f2ea00d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2ea00d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 >> v1:172.21.15.174:6789/0 conn(0x7f2ea00cff40 legacy=0x7f2ea00ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea00d1c40 con 0x7f2ea00b2050 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f2e94001670 con 0x7f2ea00b2050 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 >> v1:172.21.15.62:6789/0 conn(0x7f2ea00b2050 legacy=0x7f2ea00cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e880039e0 0x7f2e88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e88004800 0x7f2e88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2e88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8800bac0 con 0x7f2e880039e0 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8800bc70 con 0x7f2e88004800 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8800be70 con 0x7f2ea0162e40 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e880039e0 0x7f2e88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea5adb640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e88004800 0x7f2e88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea6add640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2e88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 -- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 msgr2=0x7f2e88005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2e88005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 -- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e88004800 msgr2=0x7f2e88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e88004800 0x7f2e88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea62dc640 1 -- 172.21.15.62:0/282483141 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e88003520 con 0x7f2e880039e0 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea5adb640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e88004800 0x7f2e88004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.142+0000 7f2ea6add640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea0162e40 0x7f2e88005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.143+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e880039e0 0x7f2e88003e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ac0 tx=0x7f2e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af996d964594178 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.143+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98036070 con 0x7f2e880039e0 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.143+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98031400 con 0x7f2e880039e0 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.143+0000 7f2ea52da640 1 -- 172.21.15.62:0/282483141 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98009690 con 0x7f2e880039e0 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.143+0000 7f2ea8540640 1 -- 172.21.15.62:0/282483141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e880039e0 msgr2=0x7f2e88003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea8540640 1 --2- 172.21.15.62:0/282483141 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e880039e0 0x7f2e88003e00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ac0 tx=0x7f2e88005830 comp rx=0 tx=0).stop 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea6add640 1 -- 172.21.15.62:0/282483141 reap_dead start 2024-04-26T22:27:49.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea8540640 1 -- 172.21.15.62:0/282483141 shutdown_connections 2024-04-26T22:27:49.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea8540640 1 -- 172.21.15.62:0/282483141 >> 172.21.15.62:0/282483141 conn(0x7f2ea009ba50 msgr2=0x7f2ea009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea8540640 1 -- 172.21.15.62:0/282483141 shutdown_connections 2024-04-26T22:27:49.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.144+0000 7f2ea8540640 1 -- 172.21.15.62:0/282483141 wait complete. 2024-04-26T22:27:49.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.145+0000 7f2ea8540640 1 Processor -- start 2024-04-26T22:27:49.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.145+0000 7f2ea8540640 1 -- start start 2024-04-26T22:27:49.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880039e0 0x7f2ea015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea01f5170 0x7f2ea00b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea00b24b0 con 0x7f2ea015cc90 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea00d0eb0 con 0x7f2e880039e0 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea00b2630 con 0x7f2ea01f5170 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43292/0 (socket says 172.21.15.62:43292) 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 -- 172.21.15.62:0/411233434 learned_addr learned my addr 172.21.15.62:0/411233434 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea5adb640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea01f5170 0x7f2ea00b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880039e0 0x7f2ea015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 -- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea01f5170 msgr2=0x7f2ea00b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea01f5170 0x7f2ea00b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 -- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880039e0 msgr2=0x7f2ea015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880039e0 0x7f2ea015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 -- 172.21.15.62:0/411233434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea01f5670 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea6add640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e90000f80 tx=0x7f2e9000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90006d50 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e90003970 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90003da0 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea01f5950 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.146+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea01f5f10 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.147+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea00b8cc0 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.148+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e9001b020 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.148+0000 7f2e9effd640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 0x7f2e8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.148+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e9008b5a0 con 0x7f2ea015cc90 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.148+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 0x7f2e8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.149+0000 7f2ea62dc640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 0x7f2e8c06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e98033000 tx=0x7f2e98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.152+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ea00b8cc0 con 0x7f2ea015cc90 2024-04-26T22:27:49.338 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.337+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ea006a360 con 0x7f2ea015cc90 2024-04-26T22:27:49.339 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.338+0000 7f2e9effd640 1 -- 172.21.15.62:0/411233434 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f2e90059a60 con 0x7f2ea015cc90 2024-04-26T22:27:49.339 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:49.339 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 msgr2=0x7f2e8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 0x7f2e8c06ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e98033000 tx=0x7f2e98008030 comp rx=0 tx=0).stop 2024-04-26T22:27:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 msgr2=0x7f2ea0150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.341 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2e90000f80 tx=0x7f2e9000dc00 comp rx=0 tx=0).stop 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 shutdown_connections 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2e8c06c920 0x7f2e8c06ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ea015cc90 0x7f2ea0150f90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ea01f5170 0x7f2ea00b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 --2- 172.21.15.62:0/411233434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e880039e0 0x7f2ea015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 >> 172.21.15.62:0/411233434 conn(0x7f2ea009ba50 msgr2=0x7f2ea00cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 shutdown_connections 2024-04-26T22:27:49.342 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.340+0000 7f2ea8540640 1 -- 172.21.15.62:0/411233434 wait complete. 2024-04-26T22:27:49.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:27:11.724834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:27:12.167844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:49.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.455+0000 7f3c3e8b8640 1 Processor -- start 2024-04-26T22:27:49.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.455+0000 7f3c3e8b8640 1 -- start start 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3e8b8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3e8b8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c380d1fe0 0x7f3c380d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3e8b8640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c380cd640 con 0x7f3c380b2290 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3e8b8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f3c380cd920 con 0x7f3c380d1fe0 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3e8b8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3c380cdc00 con 0x7f3c380cff80 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3ce55640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f3c380d1fe0 0x7f3c380d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c37fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3c380b2290 0x7f3c380cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60928/0 (socket says 172.21.15.62:60928) 2024-04-26T22:27:49.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c37fff640 1 -- 172.21.15.62:0/3809876288 learned_addr learned my addr 172.21.15.62:0/3809876288 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c377fe640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3ce55640 1 -- 172.21.15.62:0/3809876288 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 msgr2=0x7f3c380b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3ce55640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3ce55640 1 -- 172.21.15.62:0/3809876288 >> v1:172.21.15.62:6789/0 conn(0x7f3c380b2290 legacy=0x7f3c380cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c3ce55640 1 -- 172.21.15.62:0/3809876288 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c380cdee0 con 0x7f3c380d1fe0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.456+0000 7f3c36ffd640 1 -- 172.21.15.62:0/3809876288 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278254406 0 0) 0x7f3c380cd640 con 0x7f3c380b2290 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.457+0000 7f3c3ce55640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380d1fe0 0x7f3c380d2450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009d90 tx=0x7f3c3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3a4947d05ad089 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.457+0000 7f3c36ffd640 1 -- 172.21.15.62:0/3809876288 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30007740 con 0x7f3c380d1fe0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.457+0000 7f3c36ffd640 1 -- 172.21.15.62:0/3809876288 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c300078e0 con 0x7f3c380d1fe0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.457+0000 7f3c36ffd640 1 -- 172.21.15.62:0/3809876288 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30007cb0 con 0x7f3c380d1fe0 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3809876288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380d1fe0 msgr2=0x7f3c380d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380d1fe0 0x7f3c380d2450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009d90 tx=0x7f3c3002a660 comp rx=0 tx=0).stop 2024-04-26T22:27:49.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3809876288 shutdown_connections 2024-04-26T22:27:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380d1fe0 0x7f3c380d2450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3809876288 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3809876288 >> 172.21.15.62:0/3809876288 conn(0x7f3c380d2ff0 msgr2=0x7f3c380d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3809876288 shutdown_connections 2024-04-26T22:27:49.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.458+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3809876288 wait complete. 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.459+0000 7f3c3e8b8640 1 Processor -- start 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.459+0000 7f3c3e8b8640 1 -- start start 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 0x7f3c380cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380d2990 con 0x7f3c380b2290 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380b2670 con 0x7f3c380d1fe0 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3e8b8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3c380d2b10 con 0x7f3c380cff80 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43306/0 (socket says 172.21.15.62:43306) 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 -- 172.21.15.62:0/3926072450 learned_addr learned my addr 172.21.15.62:0/3926072450 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3ce55640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 0x7f3c380cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c377fe640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.464 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 -- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 msgr2=0x7f3c380cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 -- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 msgr2=0x7f3c380cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 0x7f3c380cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c37fff640 1 -- 172.21.15.62:0/3926072450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c380b6b30 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c3ce55640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 0x7f3c380cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.460+0000 7f3c377fe640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c37fff640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c4d0 tx=0x7f3c2000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20018070 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c20007500 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20004a20 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c380b6e10 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.461+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c381538a0 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.462+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c380ca0c0 con 0x7f3c380b2290 2024-04-26T22:27:49.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.462+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c2001b070 con 0x7f3c380b2290 2024-04-26T22:27:49.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.462+0000 7f3c34ff9640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 0x7f3c1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.463+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3c2008ae00 con 0x7f3c380b2290 2024-04-26T22:27:49.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.463+0000 7f3c377fe640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 0x7f3c1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.463+0000 7f3c377fe640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 0x7f3c1806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cc2f0 tx=0x7f3c28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.467+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c200592c0 con 0x7f3c380b2290 2024-04-26T22:27:49.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.653+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c3806a360 con 0x7f3c380b2290 2024-04-26T22:27:49.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.654+0000 7f3c34ff9640 1 -- 172.21.15.62:0/3926072450 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3c20058bc0 con 0x7f3c380b2290 2024-04-26T22:27:49.655 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.655+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 msgr2=0x7f3c1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.655+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 0x7f3c1806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c380cc2f0 tx=0x7f3c28008040 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.655+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 msgr2=0x7f3c38153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.655+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c4d0 tx=0x7f3c2000c990 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 shutdown_connections 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3c1806c8f0 0x7f3c1806edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c380d1fe0 0x7f3c380cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c380cff80 0x7f3c380cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 --2- 172.21.15.62:0/3926072450 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3c380b2290 0x7f3c38153360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 >> 172.21.15.62:0/3926072450 conn(0x7f3c380d2ff0 msgr2=0x7f3c380d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 shutdown_connections 2024-04-26T22:27:49.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.656+0000 7f3c3e8b8640 1 -- 172.21.15.62:0/3926072450 wait complete. 2024-04-26T22:27:49.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:27:49.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.770+0000 7f8bb6b19640 1 Processor -- start 2024-04-26T22:27:49.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.770+0000 7f8bb6b19640 1 -- start start 2024-04-26T22:27:49.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb6b19640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bb00b6820 0x7f8bb00c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb6b19640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 0x7f8bb00c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb6b19640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb00b5fd0 con 0x7f8bb00b5510 2024-04-26T22:27:49.772 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb6b19640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb00b5d80 con 0x7f8bb00b6820 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb6b19640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb00d5c80 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8bb00b6820 0x7f8bb00c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb48b5640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8bb00b5510 0x7f8bb00d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60934/0 (socket says 172.21.15.62:60934) 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb48b5640 1 -- 172.21.15.62:0/3097460934 learned_addr learned my addr 172.21.15.62:0/3097460934 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb50b6640 1 --2- 172.21.15.62:0/3097460934 >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 0x7f8bb00c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baffff640 1 -- 172.21.15.62:0/3097460934 >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 msgr2=0x7f8bb00c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baf7fe640 1 -- 172.21.15.62:0/3097460934 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011743089 0 0) 0x7f8bb00b5fd0 con 0x7f8bb00b5510 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baffff640 1 --2- 172.21.15.62:0/3097460934 >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 0x7f8bb00c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baffff640 1 -- 172.21.15.62:0/3097460934 >> v1:172.21.15.62:6789/0 conn(0x7f8bb00b5510 legacy=0x7f8bb00d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8baffff640 1 -- 172.21.15.62:0/3097460934 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb00d5f60 con 0x7f8bb00b6820 2024-04-26T22:27:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.771+0000 7f8bb50b6640 1 --2- 172.21.15.62:0/3097460934 >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 0x7f8bb00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8baffff640 1 --2- 172.21.15.62:0/3097460934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00b6820 0x7f8bb00c8cb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8bb008e5d0 tx=0x7f8b9c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee0813d6f098e41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8baf7fe640 1 -- 172.21.15.62:0/3097460934 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c037070 con 0x7f8bb00b6820 2024-04-26T22:27:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8baf7fe640 1 -- 172.21.15.62:0/3097460934 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c032440 con 0x7f8bb00b6820 2024-04-26T22:27:49.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8baf7fe640 1 -- 172.21.15.62:0/3097460934 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c02ae60 con 0x7f8bb00b6820 2024-04-26T22:27:49.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8bb6b19640 1 -- 172.21.15.62:0/3097460934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00b6820 msgr2=0x7f8bb00c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.772+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/3097460934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00b6820 0x7f8bb00c8cb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8bb008e5d0 tx=0x7f8b9c02a110 comp rx=0 tx=0).stop 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 -- 172.21.15.62:0/3097460934 shutdown_connections 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/3097460934 >> v2:172.21.15.174:3300/0 conn(0x7f8bb00d1fa0 0x7f8bb00c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/3097460934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00b6820 0x7f8bb00c8cb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 -- 172.21.15.62:0/3097460934 >> 172.21.15.62:0/3097460934 conn(0x7f8bb00d5160 msgr2=0x7f8bb00d5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 -- 172.21.15.62:0/3097460934 shutdown_connections 2024-04-26T22:27:49.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.773+0000 7f8bb6b19640 1 -- 172.21.15.62:0/3097460934 wait complete. 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.774+0000 7f8bb6b19640 1 Processor -- start 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.774+0000 7f8bb6b19640 1 -- start start 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 0x7f8bb01f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 0x7f8bb00a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb00d4100 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb00d0b60 con 0x7f8bb00b6820 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb6b19640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb00d1080 con 0x7f8bb00b5510 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43322/0 (socket says 172.21.15.62:43322) 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 -- 172.21.15.62:0/2049418142 learned_addr learned my addr 172.21.15.62:0/2049418142 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8baffff640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 0x7f8bb00a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb48b5640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 0x7f8bb01f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.781 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 -- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 msgr2=0x7f8bb01f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 0x7f8bb01f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 -- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 msgr2=0x7f8bb00a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 0x7f8bb00a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 -- 172.21.15.62:0/2049418142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb0152530 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8baffff640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 0x7f8bb00a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb48b5640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 0x7f8bb01f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bb50b6640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800cd10 tx=0x7f8ba800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003b90 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.775+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba8003d50 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.776+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb0152810 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.776+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004260 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.776+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb0152dd0 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.776+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb00693e0 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.777+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ba8004400 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.777+0000 7f8bad7fa640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 0x7f8b9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.777+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8ba808c560 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.778+0000 7f8bb48b5640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 0x7f8b9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.778+0000 7f8bb48b5640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 0x7f8b9406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f60 tx=0x7f8ba4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:49.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.782+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ba8014030 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.969 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.968+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8bb00d4700 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.970 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.968+0000 7f8bad7fa640 1 -- 172.21.15.62:0/2049418142 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f8ba805aa20 con 0x7f8bb00d1fa0 2024-04-26T22:27:49.970 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:27:49.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.971+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 msgr2=0x7f8b9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.971+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 0x7f8b9406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f60 tx=0x7f8ba4009040 comp rx=0 tx=0).stop 2024-04-26T22:27:49.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.971+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 msgr2=0x7f8bb006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:49.973 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.971+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800cd10 tx=0x7f8ba800a650 comp rx=0 tx=0).stop 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 shutdown_connections 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8b9406c8f0 0x7f8b9406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8bb00d1fa0 0x7f8bb006d740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bb00b6820 0x7f8bb00a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 --2- 172.21.15.62:0/2049418142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bb00b5510 0x7f8bb01f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 >> 172.21.15.62:0/2049418142 conn(0x7f8bb00d5160 msgr2=0x7f8bb00b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 shutdown_connections 2024-04-26T22:27:49.974 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:49.972+0000 7f8bb6b19640 1 -- 172.21.15.62:0/2049418142 wait complete. 2024-04-26T22:27:49.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:27:50.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.086+0000 7f3da04f8640 1 Processor -- start 2024-04-26T22:27:50.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.086+0000 7f3da04f8640 1 -- start start 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3da04f8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d980d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3da04f8640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d98161510 con 0x7f3d98160800 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3da04f8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d980d0930 con 0x7f3d981610c0 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3da04f8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3d980d0c00 con 0x7f3d980d2320 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3d9e294640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3d98160800 0x7f3d980d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60938/0 (socket says 172.21.15.62:60938) 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.087+0000 7f3d9e294640 1 -- 172.21.15.62:0/4044929693 learned_addr learned my addr 172.21.15.62:0/4044929693 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d980d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2733800353 0 0) 0x7f3d98161510 con 0x7f3d98160800 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d80003520 con 0x7f3d98160800 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2960199331 0 0) 0x7f3d80003520 con 0x7f3d98160800 2024-04-26T22:27:50.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 msgr2=0x7f3d980d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d980d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 >> v1:172.21.15.174:6789/0 conn(0x7f3d981610c0 legacy=0x7f3d980d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980d0ee0 con 0x7f3d98160800 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f3d90001670 con 0x7f3d98160800 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 >> v1:172.21.15.62:6789/0 conn(0x7f3d98160800 legacy=0x7f3d980d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80003f20 0x7f3d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d80004d40 0x7f3d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980d0c00 con 0x7f3d80004d40 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8000ba20 con 0x7f3d80003f20 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8000bc20 con 0x7f3d980d2320 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d80004d40 0x7f3d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9e294640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9da93640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80003f20 0x7f3d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 -- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 msgr2=0x7f3d800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980d2320 0x7f3d800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 -- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80003f20 msgr2=0x7f3d80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80003f20 0x7f3d80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.088+0000 7f3d9ea95640 1 -- 172.21.15.62:0/4044929693 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d80003520 con 0x7f3d80004d40 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.089+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d80004d40 0x7f3d800051c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009f90 tx=0x7f3d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bed739074c7adc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.089+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009c10 con 0x7f3d80004d40 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.089+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88036070 con 0x7f3d80004d40 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.089+0000 7f3d9d292640 1 -- 172.21.15.62:0/4044929693 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031850 con 0x7f3d80004d40 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 -- 172.21.15.62:0/4044929693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d80004d40 msgr2=0x7f3d800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 --2- 172.21.15.62:0/4044929693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d80004d40 0x7f3d800051c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009f90 tx=0x7f3d80005830 comp rx=0 tx=0).stop 2024-04-26T22:27:50.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3d9ea95640 1 -- 172.21.15.62:0/4044929693 reap_dead start 2024-04-26T22:27:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 -- 172.21.15.62:0/4044929693 shutdown_connections 2024-04-26T22:27:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 -- 172.21.15.62:0/4044929693 >> 172.21.15.62:0/4044929693 conn(0x7f3d9809e550 msgr2=0x7f3d980d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 -- 172.21.15.62:0/4044929693 shutdown_connections 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.090+0000 7f3da04f8640 1 -- 172.21.15.62:0/4044929693 wait complete. 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.091+0000 7f3da04f8640 1 Processor -- start 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.091+0000 7f3da04f8640 1 -- start start 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80004d40 0x7f3d980cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980b9cc0 0x7f3d980ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980d30f0 con 0x7f3d981f5ba0 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980d2e10 con 0x7f3d80004d40 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3da04f8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3d980b6300 con 0x7f3d980b9cc0 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43340/0 (socket says 172.21.15.62:43340) 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 -- 172.21.15.62:0/1339736224 learned_addr learned my addr 172.21.15.62:0/1339736224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9e294640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80004d40 0x7f3d980cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9da93640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980b9cc0 0x7f3d980ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 -- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980b9cc0 msgr2=0x7f3d980ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980b9cc0 0x7f3d980ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 -- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80004d40 msgr2=0x7f3d980cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80004d40 0x7f3d980cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 -- 172.21.15.62:0/1339736224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980b8d20 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.092+0000 7f3d9ea95640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009f90 tx=0x7f3d88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.093+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88009c10 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.093+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88003b90 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.093+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d980b7f60 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.093+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031e30 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.093+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d980b8520 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.094+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9806a360 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.094+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d88005d10 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.095+0000 7f3d96ffd640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 0x7f3d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.095+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d880ad210 con 0x7f3d981f5ba0 2024-04-26T22:27:50.098 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.095+0000 7f3d9e294640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 0x7f3d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.099 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.095+0000 7f3d9e294640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 0x7f3d8406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007d70 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.100 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.099+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d8804a060 con 0x7f3d981f5ba0 2024-04-26T22:27:50.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.284+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3d980d4530 con 0x7f3d981f5ba0 2024-04-26T22:27:50.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.284+0000 7f3d96ffd640 1 -- 172.21.15.62:0/1339736224 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3d8802e050 con 0x7f3d981f5ba0 2024-04-26T22:27:50.286 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 msgr2=0x7f3d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 0x7f3d8406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007d70 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 msgr2=0x7f3d981f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009f90 tx=0x7f3d88009750 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 shutdown_connections 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f3d8406c8f0 0x7f3d8406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d981f5ba0 0x7f3d981f6020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3d980b9cc0 0x7f3d980ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 --2- 172.21.15.62:0/1339736224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d80004d40 0x7f3d980cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 >> 172.21.15.62:0/1339736224 conn(0x7f3d9809e550 msgr2=0x7f3d980d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 shutdown_connections 2024-04-26T22:27:50.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.286+0000 7f3da04f8640 1 -- 172.21.15.62:0/1339736224 wait complete. 2024-04-26T22:27:50.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:50.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.397+0000 7f8c85d71640 1 Processor -- start 2024-04-26T22:27:50.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.397+0000 7f8c85d71640 1 -- start start 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c85d71640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c800cff40 0x7f8c800d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c85d71640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c800b1d70 con 0x7f8c8006ea40 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c85d71640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c800d1960 con 0x7f8c80162e40 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c85d71640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8c80163220 con 0x7f8c800cff40 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7effd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8c8006ea40 0x7f8c800cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60950/0 (socket says 172.21.15.62:60950) 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7ffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8c800cff40 0x7f8c800d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7effd640 1 -- 172.21.15.62:0/1092672772 learned_addr learned my addr 172.21.15.62:0/1092672772 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7ffff640 1 -- 172.21.15.62:0/1092672772 >> v1:172.21.15.174:6789/0 conn(0x7f8c80162e40 legacy=0x7f8c800b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7ffff640 1 -- 172.21.15.62:0/1092672772 >> v1:172.21.15.62:6789/0 conn(0x7f8c8006ea40 legacy=0x7f8c800cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7ffff640 1 -- 172.21.15.62:0/1092672772 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800d1c40 con 0x7f8c800cff40 2024-04-26T22:27:50.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.398+0000 7f8c7e7fc640 1 -- 172.21.15.62:0/1092672772 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952362699 0 0) 0x7f8c800b1d70 con 0x7f8c8006ea40 2024-04-26T22:27:50.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.399+0000 7f8c7ffff640 1 --2- 172.21.15.62:0/1092672772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800d1340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d90 tx=0x7f8c7802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703f7ee9f04e2906 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.401 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.399+0000 7f8c7e7fc640 1 -- 172.21.15.62:0/1092672772 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78007740 con 0x7f8c800cff40 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.399+0000 7f8c7e7fc640 1 -- 172.21.15.62:0/1092672772 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c780078e0 con 0x7f8c800cff40 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.399+0000 7f8c7e7fc640 1 -- 172.21.15.62:0/1092672772 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78007cb0 con 0x7f8c800cff40 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 -- 172.21.15.62:0/1092672772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 msgr2=0x7f8c800d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 --2- 172.21.15.62:0/1092672772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800d1340 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009d90 tx=0x7f8c7802a660 comp rx=0 tx=0).stop 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 -- 172.21.15.62:0/1092672772 shutdown_connections 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 --2- 172.21.15.62:0/1092672772 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800d1340 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 -- 172.21.15.62:0/1092672772 >> 172.21.15.62:0/1092672772 conn(0x7f8c8009ba50 msgr2=0x7f8c8009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 -- 172.21.15.62:0/1092672772 shutdown_connections 2024-04-26T22:27:50.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.400+0000 7f8c85d71640 1 -- 172.21.15.62:0/1092672772 wait complete. 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.401+0000 7f8c85d71640 1 Processor -- start 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.401+0000 7f8c85d71640 1 -- start start 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c8006ea40 0x7f8c8015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c80162e40 0x7f8c8015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8c800ce010 con 0x7f8c800cff40 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c800d0eb0 con 0x7f8c8006ea40 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c85d71640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c800ce190 con 0x7f8c80162e40 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43346/0 (socket says 172.21.15.62:43346) 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 -- 172.21.15.62:0/3594853519 learned_addr learned my addr 172.21.15.62:0/3594853519 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7ffff640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c80162e40 0x7f8c8015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7f7fe640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c8006ea40 0x7f8c8015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 -- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c80162e40 msgr2=0x7f8c8015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c80162e40 0x7f8c8015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 -- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c8006ea40 msgr2=0x7f8c8015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c8006ea40 0x7f8c8015a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.402+0000 7f8c7effd640 1 -- 172.21.15.62:0/3594853519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800c7490 con 0x7f8c800cff40 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c7effd640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c4a0 tx=0x7f8c7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74018070 con 0x7f8c800cff40 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c740074e0 con 0x7f8c800cff40 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c800c7770 con 0x7f8c800cff40 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004a20 con 0x7f8c800cff40 2024-04-26T22:27:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.403+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c80160e80 con 0x7f8c800cff40 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.404+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8006a360 con 0x7f8c800cff40 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.404+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c7401b070 con 0x7f8c800cff40 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.405+0000 7f8c67fff640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 0x7f8c6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.405+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c7408aea0 con 0x7f8c800cff40 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.405+0000 7f8c7f7fe640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 0x7f8c6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.405+0000 7f8c7f7fe640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 0x7f8c6c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c70000f60 tx=0x7f8c70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.409+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c740592d0 con 0x7f8c800cff40 2024-04-26T22:27:50.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.636+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c8006d310 con 0x7f8c800cff40 2024-04-26T22:27:50.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.637+0000 7f8c67fff640 1 -- 172.21.15.62:0/3594853519 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f8c74058bd0 con 0x7f8c800cff40 2024-04-26T22:27:50.638 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:50.638 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:50.638 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.638+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 msgr2=0x7f8c6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.638+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 0x7f8c6c06edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c70000f60 tx=0x7f8c70009040 comp rx=0 tx=0).stop 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.638+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 msgr2=0x7f8c800c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.638+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c4a0 tx=0x7f8c7400c960 comp rx=0 tx=0).stop 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 shutdown_connections 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8c6c06c8f0 0x7f8c6c06edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c80162e40 0x7f8c8015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8c800cff40 0x7f8c800c6a10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 --2- 172.21.15.62:0/3594853519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c8006ea40 0x7f8c8015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 >> 172.21.15.62:0/3594853519 conn(0x7f8c8009ba50 msgr2=0x7f8c800cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 shutdown_connections 2024-04-26T22:27:50.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.639+0000 7f8c85d71640 1 -- 172.21.15.62:0/3594853519 wait complete. 2024-04-26T22:27:50.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.751+0000 7f503dd9e640 1 Processor -- start 2024-04-26T22:27:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.751+0000 7f503dd9e640 1 -- start start 2024-04-26T22:27:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.751+0000 7f503dd9e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f503815f2f0 0x7f50380d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.751+0000 7f503dd9e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50380d0b20 con 0x7f50380d2d50 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503dd9e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50380d0e00 con 0x7f50380b5e90 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503dd9e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f50380b4680 con 0x7f503815f2f0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503cd9c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f503815f2f0 0x7f50380d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f5037fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f50380d2d50 0x7f50380d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60954/0 (socket says 172.21.15.62:60954) 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f5037fff640 1 -- 172.21.15.62:0/1058428011 learned_addr learned my addr 172.21.15.62:0/1058428011 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503cd9c640 1 -- 172.21.15.62:0/1058428011 >> v1:172.21.15.174:6789/0 conn(0x7f50380b5e90 legacy=0x7f50380d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503cd9c640 1 -- 172.21.15.62:0/1058428011 >> v1:172.21.15.62:6789/0 conn(0x7f50380d2d50 legacy=0x7f50380d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f503cd9c640 1 -- 172.21.15.62:0/1058428011 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50380b4960 con 0x7f503815f2f0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.752+0000 7f50377fe640 1 -- 172.21.15.62:0/1058428011 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860358333 0 0) 0x7f50380d0b20 con 0x7f50380d2d50 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503cd9c640 1 --2- 172.21.15.62:0/1058428011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380d3450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5030009d90 tx=0x7f503002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bd1e3cbd25e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f50377fe640 1 -- 172.21.15.62:0/1058428011 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030007740 con 0x7f503815f2f0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f50377fe640 1 -- 172.21.15.62:0/1058428011 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50300078e0 con 0x7f503815f2f0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f50377fe640 1 -- 172.21.15.62:0/1058428011 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030007cb0 con 0x7f503815f2f0 2024-04-26T22:27:50.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 -- 172.21.15.62:0/1058428011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 msgr2=0x7f50380d3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1058428011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380d3450 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5030009d90 tx=0x7f503002a6a0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 -- 172.21.15.62:0/1058428011 shutdown_connections 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1058428011 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380d3450 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 -- 172.21.15.62:0/1058428011 >> 172.21.15.62:0/1058428011 conn(0x7f50380d4f70 msgr2=0x7f50380d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 -- 172.21.15.62:0/1058428011 shutdown_connections 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.753+0000 7f503dd9e640 1 -- 172.21.15.62:0/1058428011 wait complete. 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.755+0000 7f503dd9e640 1 Processor -- start 2024-04-26T22:27:50.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.755+0000 7f503dd9e640 1 -- start start 2024-04-26T22:27:50.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 0x7f50380ba690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 0x7f50381f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f503815f850 con 0x7f503815f2f0 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f503815f6d0 con 0x7f50380b5e90 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f50380d2570 con 0x7f50380d2d50 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43368/0 (socket says 172.21.15.62:43368) 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 -- 172.21.15.62:0/1633574767 learned_addr learned my addr 172.21.15.62:0/1633574767 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f5037fff640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 0x7f50381f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503cd9c640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 0x7f50380ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 -- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 msgr2=0x7f50381f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 0x7f50381f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 -- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 msgr2=0x7f50380ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 0x7f50380ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 -- 172.21.15.62:0/1633574767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381f6140 con 0x7f503815f2f0 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f5037fff640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 0x7f50381f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:50.761 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503cd9c640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 0x7f50380ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503d59d640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f502800c440 tx=0x7f502800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028011790 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5028017070 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50380ce3e0 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028011b70 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.756+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50380b7e30 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.757+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5038076890 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.758+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5028011d10 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.758+0000 7f50357fa640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.758+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f502808b930 con 0x7f503815f2f0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.759+0000 7f503cd9c640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:50.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.759+0000 7f503cd9c640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 0x7f501806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5030034000 tx=0x7f503002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:50.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.763+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5028059d60 con 0x7f503815f2f0 2024-04-26T22:27:50.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.947+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f503815e940 con 0x7f503815f2f0 2024-04-26T22:27:50.949 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.948+0000 7f50357fa640 1 -- 172.21.15.62:0/1633574767 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f5028059660 con 0x7f503815f2f0 2024-04-26T22:27:50.950 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:50.950 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 msgr2=0x7f501806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 0x7f501806edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5030034000 tx=0x7f503002f000 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 msgr2=0x7f50380cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f502800c440 tx=0x7f502800c900 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 shutdown_connections 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f501806c8f0 0x7f501806edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f503815f2f0 0x7f50380cdea0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50380d2d50 0x7f50381f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 --2- 172.21.15.62:0/1633574767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50380b5e90 0x7f50380ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 >> 172.21.15.62:0/1633574767 conn(0x7f50380d4f70 msgr2=0x7f50380d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 shutdown_connections 2024-04-26T22:27:50.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:50.950+0000 7f503dd9e640 1 -- 172.21.15.62:0/1633574767 wait complete. 2024-04-26T22:27:50.963 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T22:27:50.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:51.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.062+0000 7fed4a50a640 1 Processor -- start 2024-04-26T22:27:51.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.062+0000 7fed4a50a640 1 -- start start 2024-04-26T22:27:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed4a50a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fed440b3c90 0x7fed44168150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed4a50a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed440cf740 con 0x7fed44160a00 2024-04-26T22:27:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed4a50a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed440b2400 con 0x7fed440b1f90 2024-04-26T22:27:51.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed4a50a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fed440d07e0 con 0x7fed440b3c90 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed43fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fed44160a00 0x7fed440cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60962/0 (socket says 172.21.15.62:60962) 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed43fff640 1 -- 172.21.15.62:0/4150913864 learned_addr learned my addr 172.21.15.62:0/4150913864 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.063+0000 7fed48aa7640 1 --2- 172.21.15.62:0/4150913864 >> v2:172.21.15.174:3300/0 conn(0x7fed440b3c90 0x7fed44168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.064+0000 7fed42ffd640 1 -- 172.21.15.62:0/4150913864 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1666156488 0 0) 0x7fed440cf740 con 0x7fed44160a00 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.064+0000 7fed42ffd640 1 -- 172.21.15.62:0/4150913864 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2c003520 con 0x7fed44160a00 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.064+0000 7fed48aa7640 1 -- 172.21.15.62:0/4150913864 >> v1:172.21.15.174:6789/0 conn(0x7fed440b1f90 legacy=0x7fed440d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.064+0000 7fed48aa7640 1 -- 172.21.15.62:0/4150913864 >> v1:172.21.15.62:6789/0 conn(0x7fed44160a00 legacy=0x7fed440cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.064+0000 7fed48aa7640 1 -- 172.21.15.62:0/4150913864 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440cf080 con 0x7fed440b3c90 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed48aa7640 1 --2- 172.21.15.62:0/4150913864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed44168150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009d90 tx=0x7fed3c02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17abb27c33f7a76c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed42ffd640 1 -- 172.21.15.62:0/4150913864 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c037070 con 0x7fed440b3c90 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed42ffd640 1 -- 172.21.15.62:0/4150913864 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c007740 con 0x7fed440b3c90 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed42ffd640 1 -- 172.21.15.62:0/4150913864 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c007b10 con 0x7fed440b3c90 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 -- 172.21.15.62:0/4150913864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 msgr2=0x7fed44168150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 --2- 172.21.15.62:0/4150913864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed44168150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009d90 tx=0x7fed3c02a780 comp rx=0 tx=0).stop 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 -- 172.21.15.62:0/4150913864 shutdown_connections 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 --2- 172.21.15.62:0/4150913864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed44168150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 -- 172.21.15.62:0/4150913864 >> 172.21.15.62:0/4150913864 conn(0x7fed4409ba50 msgr2=0x7fed4409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 -- 172.21.15.62:0/4150913864 shutdown_connections 2024-04-26T22:27:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.065+0000 7fed4a50a640 1 -- 172.21.15.62:0/4150913864 wait complete. 2024-04-26T22:27:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 Processor -- start 2024-04-26T22:27:51.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 -- start start 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed440c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 0x7fed440c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed440ceda0 con 0x7fed440b1f90 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed440ceac0 con 0x7fed440b3c90 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.067+0000 7fed4a50a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fed44160de0 con 0x7fed44160a00 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43372/0 (socket says 172.21.15.62:43372) 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 -- 172.21.15.62:0/1120696941 learned_addr learned my addr 172.21.15.62:0/1120696941 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed437fe640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed440c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed48aa7640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 0x7fed440c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 -- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 msgr2=0x7fed440c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 0x7fed440c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 -- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 msgr2=0x7fed440c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed440c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 -- 172.21.15.62:0/1120696941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440b50d0 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed48aa7640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 0x7fed440c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed43fff640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed3800c530 tx=0x7fed3800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38017070 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38005ce0 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed440b6ab0 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed380072b0 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.068+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed440b66f0 con 0x7fed440b1f90 2024-04-26T22:27:51.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.069+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed4406a360 con 0x7fed440b1f90 2024-04-26T22:27:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.070+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed3801a070 con 0x7fed440b1f90 2024-04-26T22:27:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.070+0000 7fed40ff9640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 0x7fed3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.070+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed3808af00 con 0x7fed440b1f90 2024-04-26T22:27:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.070+0000 7fed437fe640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 0x7fed3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.071+0000 7fed437fe640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 0x7fed3006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed34007fd0 tx=0x7fed34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.075+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed38059480 con 0x7fed440b1f90 2024-04-26T22:27:51.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.297+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed440d2590 con 0x7fed440b1f90 2024-04-26T22:27:51.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.298+0000 7fed40ff9640 1 -- 172.21.15.62:0/1120696941 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7fed38058d80 con 0x7fed440b1f90 2024-04-26T22:27:51.299 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:51.299 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:51.300 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 msgr2=0x7fed3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 0x7fed3006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed34007fd0 tx=0x7fed34009040 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 msgr2=0x7fed440b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed3800c530 tx=0x7fed3800c9f0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 shutdown_connections 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fed3006c8f0 0x7fed3006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed44160a00 0x7fed440c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed440b3c90 0x7fed440c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 --2- 172.21.15.62:0/1120696941 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed440b1f90 0x7fed440b4a80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 >> 172.21.15.62:0/1120696941 conn(0x7fed4409ba50 msgr2=0x7fed440b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 shutdown_connections 2024-04-26T22:27:51.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.300+0000 7fed4a50a640 1 -- 172.21.15.62:0/1120696941 wait complete. 2024-04-26T22:27:51.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:27:51.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.411+0000 7f3042d46640 1 Processor -- start 2024-04-26T22:27:51.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.411+0000 7f3042d46640 1 -- start start 2024-04-26T22:27:51.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3042d46640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f303c160d70 0x7f303c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3042d46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c0b5b00 0x7f303c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3042d46640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f303c161b80 con 0x7f303c06f790 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3042d46640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f303c0b4400 con 0x7f303c160d70 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3042d46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f303c0b4120 con 0x7f303c0b5b00 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3040ae2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f303c06f790 0x7f303c074030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60976/0 (socket says 172.21.15.62:60976) 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3040ae2640 1 -- 172.21.15.62:0/3338470148 learned_addr learned my addr 172.21.15.62:0/3338470148 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3033fff640 1 --2- 172.21.15.62:0/3338470148 >> v2:172.21.15.174:3300/0 conn(0x7f303c160d70 0x7f303c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f30412e3640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c0b5b00 0x7f303c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f30337fe640 1 -- 172.21.15.62:0/3338470148 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545105717 0 0) 0x7f303c161b80 con 0x7f303c06f790 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f30337fe640 1 -- 172.21.15.62:0/3338470148 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3024003520 con 0x7f303c06f790 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3033fff640 1 -- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c0b5b00 msgr2=0x7f303c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:51.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3033fff640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c0b5b00 0x7f303c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3033fff640 1 -- 172.21.15.62:0/3338470148 >> v1:172.21.15.62:6789/0 conn(0x7f303c06f790 legacy=0x7f303c074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.412+0000 7f3033fff640 1 -- 172.21.15.62:0/3338470148 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c072ff0 con 0x7f303c160d70 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.413+0000 7f3033fff640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c160d70 0x7f303c161200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097a0 tx=0x7f302c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00e7786ab5b11bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.413+0000 7f30337fe640 1 -- 172.21.15.62:0/3338470148 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c037070 con 0x7f303c160d70 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.413+0000 7f30337fe640 1 -- 172.21.15.62:0/3338470148 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f302c032440 con 0x7f303c160d70 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.413+0000 7f30337fe640 1 -- 172.21.15.62:0/3338470148 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c032810 con 0x7f303c160d70 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 -- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c160d70 msgr2=0x7f303c161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c160d70 0x7f303c161200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f302c0097a0 tx=0x7f302c02a180 comp rx=0 tx=0).stop 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 -- 172.21.15.62:0/3338470148 shutdown_connections 2024-04-26T22:27:51.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c160d70 0x7f303c161200 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 --2- 172.21.15.62:0/3338470148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c0b5b00 0x7f303c0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 -- 172.21.15.62:0/3338470148 >> 172.21.15.62:0/3338470148 conn(0x7f303c0d0d50 msgr2=0x7f303c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 -- 172.21.15.62:0/3338470148 shutdown_connections 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.414+0000 7f3042d46640 1 -- 172.21.15.62:0/3338470148 wait complete. 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.415+0000 7f3042d46640 1 Processor -- start 2024-04-26T22:27:51.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.415+0000 7f3042d46640 1 -- start start 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 0x7f303c1441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 0x7f303c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0b3d80 con 0x7f303c160d70 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f303c0735b0 con 0x7f303c0b5b00 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3042d46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f303c0b3f00 con 0x7f303c06f790 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f30412e3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f30412e3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43380/0 (socket says 172.21.15.62:43380) 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f30412e3640 1 -- 172.21.15.62:0/3186780758 learned_addr learned my addr 172.21.15.62:0/3186780758 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3040ae2640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 0x7f303c1441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f3033fff640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 0x7f303c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.416+0000 7f30412e3640 1 -- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 msgr2=0x7f303c1441b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30412e3640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 0x7f303c1441b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30412e3640 1 -- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 msgr2=0x7f303c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30412e3640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 0x7f303c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30412e3640 1 -- 172.21.15.62:0/3186780758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c15bb90 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f3040ae2640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 0x7f303c1441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f3033fff640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 0x7f303c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30412e3640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3034009dd0 tx=0x7f303400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034018020 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3034014480 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303c15b8b0 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034014870 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.417+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303c1592a0 con 0x7f303c160d70 2024-04-26T22:27:51.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.418+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303c06a360 con 0x7f303c160d70 2024-04-26T22:27:51.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.418+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3034005d10 con 0x7f303c160d70 2024-04-26T22:27:51.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.419+0000 7f30317fa640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 0x7f302806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.419+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f303408c640 con 0x7f303c160d70 2024-04-26T22:27:51.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.419+0000 7f3040ae2640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 0x7f302806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.421 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.420+0000 7f3040ae2640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 0x7f302806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3038005880 tx=0x7f3038009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.423+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3034022070 con 0x7f303c160d70 2024-04-26T22:27:51.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.648+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f303c160740 con 0x7f303c160d70 2024-04-26T22:27:51.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.648+0000 7f30317fa640 1 -- 172.21.15.62:0/3186780758 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5178 (secure 0 0 0) 0x7f3034010040 con 0x7f303c160d70 2024-04-26T22:27:51.650 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:51.650 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/810946815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":810946815},{"type":"v1","addr":"172.21.15.174:6836","nonce":810946815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/2454945434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":2454945434},{"type":"v1","addr":"172.21.15.174:6837","nonce":2454945434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1596176100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1596176100},{"type":"v1","addr":"172.21.15.62:6834","nonce":1596176100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:27:13.221716+0000","modified":"2024-04-26T22:27:16.231055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6835/3095900185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3095900185},{"type":"v1","addr":"172.21.15.62:6835","nonce":3095900185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4287,"qdb_cluster":[4287]},"id":1}]} 2024-04-26T22:27:51.650 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 7 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 msgr2=0x7f302806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 0x7f302806edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3038005880 tx=0x7f3038009040 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 msgr2=0x7f303c0aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3034009dd0 tx=0x7f303400c490 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 shutdown_connections 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f302806c8f0 0x7f302806edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f303c160d70 0x7f303c0aaf60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f303c0b5b00 0x7f303c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 --2- 172.21.15.62:0/3186780758 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f303c06f790 0x7f303c1441b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.650+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 >> 172.21.15.62:0/3186780758 conn(0x7f303c0d0d50 msgr2=0x7f303c0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.651+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 shutdown_connections 2024-04-26T22:27:51.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.651+0000 7f3042d46640 1 -- 172.21.15.62:0/3186780758 wait complete. 2024-04-26T22:27:51.663 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:51.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.763+0000 7feef94a9640 1 Processor -- start 2024-04-26T22:27:51.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.763+0000 7feef94a9640 1 -- start start 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef94a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feef40d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef94a9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef40d2670 con 0x7feef40d3c80 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef94a9640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef40d0210 con 0x7feef40d3820 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef94a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feef40d04f0 con 0x7feef4162b40 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef37fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7feef40d3c80 0x7feef40d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60992/0 (socket says 172.21.15.62:60992) 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef37fe640 1 -- 172.21.15.62:0/3990590916 learned_addr learned my addr 172.21.15.62:0/3990590916 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef2ffd640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feef40d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789703756 0 0) 0x7feef40d2670 con 0x7feef40d3c80 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.764+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feee0003520 con 0x7feef40d3c80 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3745950758 0 0) 0x7feee0003520 con 0x7feef40d3c80 2024-04-26T22:27:51.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 msgr2=0x7feef40d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feef40d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 >> v1:172.21.15.174:6789/0 conn(0x7feef40d3820 legacy=0x7feef40d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef40d07d0 con 0x7feef40d3c80 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7feeec0019f0 con 0x7feef40d3c80 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 >> v1:172.21.15.62:6789/0 conn(0x7feef40d3c80 legacy=0x7feef40d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee00039e0 0x7feee0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feee0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feee00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feef40d04f0 con 0x7feee0004390 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feee000b9c0 con 0x7feee00039e0 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feee000bbc0 con 0x7feef4162b40 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feee0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef2ffd640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee00039e0 0x7feee0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef37fe640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feee00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 -- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 msgr2=0x7feee00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef4162b40 0x7feee00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 -- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee00039e0 msgr2=0x7feee0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee00039e0 0x7feee0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.765+0000 7feef27fc640 1 -- 172.21.15.62:0/3990590916 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee0003520 con 0x7feee0004390 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef27fc640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feee0004860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feee4001860 tx=0x7feee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6d9d2b431ecd11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4037070 con 0x7feee0004390 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee402abf0 con 0x7feee0004390 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef1ffb640 1 -- 172.21.15.62:0/3990590916 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4001430 con 0x7feee0004390 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef94a9640 1 -- 172.21.15.62:0/3990590916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 msgr2=0x7feee0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef94a9640 1 --2- 172.21.15.62:0/3990590916 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feee0004860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feee4001860 tx=0x7feee0005830 comp rx=0 tx=0).stop 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.766+0000 7feef37fe640 1 -- 172.21.15.62:0/3990590916 reap_dead start 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.767+0000 7feef94a9640 1 -- 172.21.15.62:0/3990590916 shutdown_connections 2024-04-26T22:27:51.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.767+0000 7feef94a9640 1 -- 172.21.15.62:0/3990590916 >> 172.21.15.62:0/3990590916 conn(0x7feef409e550 msgr2=0x7feef40d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.767+0000 7feef94a9640 1 -- 172.21.15.62:0/3990590916 shutdown_connections 2024-04-26T22:27:51.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.767+0000 7feef94a9640 1 -- 172.21.15.62:0/3990590916 wait complete. 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.768+0000 7feef94a9640 1 Processor -- start 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.768+0000 7feef94a9640 1 -- start start 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef415d9c0 0x7feef41510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef40b9400 0x7feef41f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feef40d4110 con 0x7feee0004390 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feef40d0d90 con 0x7feef40b9400 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feef40b5bb0 con 0x7feef415d9c0 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43400/0 (socket says 172.21.15.62:43400) 2024-04-26T22:27:51.776 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 -- 172.21.15.62:0/1595725568 learned_addr learned my addr 172.21.15.62:0/1595725568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 -- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef415d9c0 msgr2=0x7feef41510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef415d9c0 0x7feef41510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 -- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef40b9400 msgr2=0x7feef41f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef40b9400 0x7feef41f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 -- 172.21.15.62:0/1595725568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef414e530 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef2ffd640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feedc011910 tx=0x7feedc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc00fa10 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feedc00fbb0 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef40b8080 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc014440 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.769+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef40b8920 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.771+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef406a360 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.771+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feedc012040 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.771+0000 7feeeb7fe640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.771+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feedc08c080 con 0x7feee0004390 2024-04-26T22:27:51.777 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.772+0000 7feef27fc640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.772+0000 7feef27fc640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feee4001860 tx=0x7feee402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:51.778 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.776+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feedc019030 con 0x7feee0004390 2024-04-26T22:27:51.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.961+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feef4161e10 con 0x7feee0004390 2024-04-26T22:27:51.964 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.963+0000 7feeeb7fe640 1 -- 172.21.15.62:0/1595725568 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7feedc05a4b0 con 0x7feee0004390 2024-04-26T22:27:51.964 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:51.964 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":20,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:14.858434+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:27:11.724834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T22:27:12.167844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.964+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 msgr2=0x7feed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.964+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feee4001860 tx=0x7feee402e000 comp rx=0 tx=0).stop 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.964+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 msgr2=0x7feef414dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.964+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feedc011910 tx=0x7feedc011dd0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 shutdown_connections 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef40b9400 0x7feef41f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feef415d9c0 0x7feef41510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 --2- 172.21.15.62:0/1595725568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee0004390 0x7feef414dff0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:51.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 >> 172.21.15.62:0/1595725568 conn(0x7feef409e550 msgr2=0x7feef406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:51.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 shutdown_connections 2024-04-26T22:27:51.967 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:51.965+0000 7feef94a9640 1 -- 172.21.15.62:0/1595725568 wait complete. 2024-04-26T22:27:51.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T22:27:52.078 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.077+0000 7fe403c83640 1 Processor -- start 2024-04-26T22:27:52.079 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.078+0000 7fe403c83640 1 -- start start 2024-04-26T22:27:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.078+0000 7fe403c83640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3fc161200 0x7fe3fc0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.078+0000 7fe403c83640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3fc161b80 con 0x7fe3fc06f790 2024-04-26T22:27:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.078+0000 7fe403c83640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3fc161690 con 0x7fe3fc160d70 2024-04-26T22:27:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.078+0000 7fe403c83640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b4400 con 0x7fe3fc161200 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe401a1f640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe3fc06f790 0x7fe3fc074030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:60998/0 (socket says 172.21.15.62:60998) 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe401a1f640 1 -- 172.21.15.62:0/1028443735 learned_addr learned my addr 172.21.15.62:0/1028443735 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1367113673 0 0) 0x7fe3fc161b80 con 0x7fe3fc06f790 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e8003520 con 0x7fe3fc06f790 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe402220640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3fc161200 0x7fe3fc0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3644763189 0 0) 0x7fe3e8003520 con 0x7fe3fc06f790 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3fc161200 msgr2=0x7fe3fc0b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3fc161200 0x7fe3fc0b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 >> v1:172.21.15.174:6789/0 conn(0x7fe3fc160d70 legacy=0x7fe3fc0b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc0b4120 con 0x7fe3fc06f790 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fe3f8001670 con 0x7fe3fc06f790 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 >> v1:172.21.15.62:6789/0 conn(0x7fe3fc06f790 legacy=0x7fe3fc074030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e80039e0 0x7fe3e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e8004800 0x7fe3e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3e80051c0 0x7fe3e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b4400 con 0x7fe3e80039e0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e800c090 con 0x7fe3e80051c0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e800c290 con 0x7fe3e8004800 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe402220640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3fc161200 0x7fe3fc0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e80039e0 0x7fe3e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe40121e640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e8004800 0x7fe3e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.079+0000 7fe402220640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3e80051c0 0x7fe3e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 -- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e8004800 msgr2=0x7fe3e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e8004800 0x7fe3e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 -- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3e80051c0 msgr2=0x7fe3e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3e80051c0 0x7fe3e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 -- 172.21.15.62:0/1028443735 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e8003520 con 0x7fe3e80039e0 2024-04-26T22:27:52.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe40121e640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e8004800 0x7fe3e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.080+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e80039e0 0x7fe3e8003e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c210 tx=0x7fe3e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f799f32559f3e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007d30 con 0x7fe3e80039e0 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f8005ad0 con 0x7fe3e80039e0 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe400a1d640 1 -- 172.21.15.62:0/1028443735 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8007070 con 0x7fe3e80039e0 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 -- 172.21.15.62:0/1028443735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e80039e0 msgr2=0x7fe3e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 --2- 172.21.15.62:0/1028443735 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3e80039e0 0x7fe3e8003e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c210 tx=0x7fe3e8005830 comp rx=0 tx=0).stop 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe402220640 1 -- 172.21.15.62:0/1028443735 reap_dead start 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 -- 172.21.15.62:0/1028443735 shutdown_connections 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 -- 172.21.15.62:0/1028443735 >> 172.21.15.62:0/1028443735 conn(0x7fe3fc0d0d50 msgr2=0x7fe3fc0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 -- 172.21.15.62:0/1028443735 shutdown_connections 2024-04-26T22:27:52.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.081+0000 7fe403c83640 1 -- 172.21.15.62:0/1028443735 wait complete. 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.082+0000 7fe403c83640 1 Processor -- start 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.082+0000 7fe403c83640 1 -- start start 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e80039e0 0x7fe3fc140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 0x7fe3fc1420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0735b0 con 0x7fe3fc158200 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3fc0732d0 con 0x7fe3fc141c20 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe403c83640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe3fc0b3d80 con 0x7fe3e80039e0 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe40121e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe40121e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43424/0 (socket says 172.21.15.62:43424) 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe40121e640 1 -- 172.21.15.62:0/1437447036 learned_addr learned my addr 172.21.15.62:0/1437447036 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:52.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe402220640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 0x7fe3fc1420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.083+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e80039e0 0x7fe3fc140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 -- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e80039e0 msgr2=0x7fe3fc140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e80039e0 0x7fe3fc140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 -- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 msgr2=0x7fe3fc1420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 0x7fe3fc1420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 -- 172.21.15.62:0/1437447036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3fc158680 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe402220640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 0x7fe3fc1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe40121e640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e550 tx=0x7fe3ec00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0052e0 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec018070 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3fc140b80 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec003830 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.084+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3fc141140 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.085+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3fc06a360 con 0x7fe3fc158200 2024-04-26T22:27:52.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.085+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3ec0039d0 con 0x7fe3fc158200 2024-04-26T22:27:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.086+0000 7fe3f27fc640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 0x7fe3e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.086+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe3ec08b7e0 con 0x7fe3fc158200 2024-04-26T22:27:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.086+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 0x7fe3e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.087+0000 7fe401a1f640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 0x7fe3e006ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c210 tx=0x7fe3f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.090+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3ec059ca0 con 0x7fe3fc158200 2024-04-26T22:27:52.317 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.316+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe3fc145ff0 con 0x7fe3fc158200 2024-04-26T22:27:52.317 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-26T22:27:52.316+0000 7facc72cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:27:52.319 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.318+0000 7fe3f27fc640 1 -- 172.21.15.62:0/1437447036 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fe3ec0595a0 con 0x7fe3fc158200 2024-04-26T22:27:52.319 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 msgr2=0x7fe3e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 0x7fe3e006ede0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800c210 tx=0x7fe3f802e000 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 msgr2=0x7fe3fc1454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00e550 tx=0x7fe3ec00ea10 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 shutdown_connections 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe3e006c920 0x7fe3e006ede0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3fc141c20 0x7fe3fc1420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3fc158200 0x7fe3fc1454e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 --2- 172.21.15.62:0/1437447036 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe3e80039e0 0x7fe3fc140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.321+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 >> 172.21.15.62:0/1437447036 conn(0x7fe3fc0d0d50 msgr2=0x7fe3fc0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:52.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.322+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 shutdown_connections 2024-04-26T22:27:52.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.322+0000 7fe403c83640 1 -- 172.21.15.62:0/1437447036 wait complete. 2024-04-26T22:27:52.328 INFO:tasks.ceph.mds.c.smithi062.stderr: -7341> 2024-04-26T22:27:19.216+0000 7f29a69c4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:27:52.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T22:27:52.364 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-26T22:27:52.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.442+0000 7f8a30a07640 1 Processor -- start 2024-04-26T22:27:52.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.442+0000 7f8a30a07640 1 -- start start 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a30a07640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 0x7f8a2c0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a30a07640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a2c15f940 con 0x7f8a2c0d10a0 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a30a07640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a2c0d43a0 con 0x7f8a2c0d23a0 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a30a07640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c0d3300 con 0x7f8a2c15f560 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a23fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8a2c0d10a0 0x7f8a2c0d1c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:32778/0 (socket says 172.21.15.62:32778) 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.443+0000 7f8a23fff640 1 -- 172.21.15.62:0/691718879 learned_addr learned my addr 172.21.15.62:0/691718879 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a2a59c640 1 --2- 172.21.15.62:0/691718879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 0x7f8a2c0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 553237808 0 0) 0x7f8a2c15f940 con 0x7f8a2c0d10a0 2024-04-26T22:27:52.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a18003520 con 0x7f8a2c0d10a0 2024-04-26T22:27:52.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a2a59c640 1 -- 172.21.15.62:0/691718879 >> v1:172.21.15.174:6789/0 conn(0x7f8a2c0d23a0 legacy=0x7f8a2c1693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2750963534 0 0) 0x7f8a2c0d43a0 con 0x7f8a2c0d23a0 2024-04-26T22:27:52.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a2a59c640 1 -- 172.21.15.62:0/691718879 >> v1:172.21.15.62:6789/0 conn(0x7f8a2c0d10a0 legacy=0x7f8a2c0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.446 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.444+0000 7f8a2a59c640 1 -- 172.21.15.62:0/691718879 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0d35e0 con 0x7f8a2c15f560 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a2a59c640 1 --2- 172.21.15.62:0/691718879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 0x7f8a2c0d0b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097d0 tx=0x7f8a1c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16f7b80166f9a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c037070 con 0x7f8a2c15f560 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c032440 con 0x7f8a2c15f560 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a29d9b640 1 -- 172.21.15.62:0/691718879 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0327f0 con 0x7f8a2c15f560 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 -- 172.21.15.62:0/691718879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 msgr2=0x7f8a2c0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 --2- 172.21.15.62:0/691718879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 0x7f8a2c0d0b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0097d0 tx=0x7f8a1c02a210 comp rx=0 tx=0).stop 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 -- 172.21.15.62:0/691718879 shutdown_connections 2024-04-26T22:27:52.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 --2- 172.21.15.62:0/691718879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c15f560 0x7f8a2c0d0b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 -- 172.21.15.62:0/691718879 >> 172.21.15.62:0/691718879 conn(0x7f8a2c09e570 msgr2=0x7f8a2c0d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:52.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 -- 172.21.15.62:0/691718879 shutdown_connections 2024-04-26T22:27:52.448 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.445+0000 7f8a30a07640 1 -- 172.21.15.62:0/691718879 wait complete. 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.447+0000 7f8a30a07640 1 Processor -- start 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.447+0000 7f8a30a07640 1 -- start start 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.447+0000 7f8a30a07640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c0d10a0 0x7f8a2c156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a2c0d23a0 0x7f8a2c0b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0d3e80 con 0x7f8a2c15f560 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0d3ba0 con 0x7f8a2c0d23a0 2024-04-26T22:27:52.450 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c0b6ac0 con 0x7f8a2c0d10a0 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43440/0 (socket says 172.21.15.62:43440) 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 -- 172.21.15.62:0/1887915121 learned_addr learned my addr 172.21.15.62:0/1887915121 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2a59c640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c0d10a0 0x7f8a2c156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a23fff640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a2c0d23a0 0x7f8a2c0b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 -- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c0d10a0 msgr2=0x7f8a2c156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c0d10a0 0x7f8a2c156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 -- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a2c0d23a0 msgr2=0x7f8a2c0b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a2c0d23a0 0x7f8a2c0b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 -- 172.21.15.62:0/1887915121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0bab50 con 0x7f8a2c15f560 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a2ad9d640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c780 tx=0x7f8a2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24003b70 con 0x7f8a2c15f560 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a24003d30 con 0x7f8a2c15f560 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c0cc450 con 0x7f8a2c15f560 2024-04-26T22:27:52.451 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.448+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a240052b0 con 0x7f8a2c15f560 2024-04-26T22:27:52.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.449+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c0cca10 con 0x7f8a2c15f560 2024-04-26T22:27:52.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.450+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c06a360 con 0x7f8a2c15f560 2024-04-26T22:27:52.452 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.450+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a24005450 con 0x7f8a2c15f560 2024-04-26T22:27:52.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.451+0000 7f8a22ffd640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 0x7f8a0c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:52.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.451+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f8a2408b8d0 con 0x7f8a2c15f560 2024-04-26T22:27:52.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.451+0000 7f8a2a59c640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 0x7f8a0c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:52.453 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.451+0000 7f8a2a59c640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 0x7f8a0c06ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:52.456 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.455+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a240595d0 con 0x7f8a2c15f560 2024-04-26T22:27:52.687 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:52.686+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8a2c069600 con 0x7f8a2c15f560 2024-04-26T22:27:53.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.319+0000 7f8a22ffd640 1 -- 172.21.15.62:0/1887915121 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f8a2405d8d0 con 0x7f8a2c15f560 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 msgr2=0x7f8a0c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 0x7f8a0c06ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c034000 tx=0x7f8a1c02e000 comp rx=0 tx=0).stop 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 msgr2=0x7f8a2c0ba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c780 tx=0x7f8a2400cc40 comp rx=0 tx=0).stop 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 shutdown_connections 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8a0c06c920 0x7f8a0c06ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a2c15f560 0x7f8a2c0ba140 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a2c0d23a0 0x7f8a2c0b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 --2- 172.21.15.62:0/1887915121 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a2c0d10a0 0x7f8a2c156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 >> 172.21.15.62:0/1887915121 conn(0x7f8a2c09e570 msgr2=0x7f8a2c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:53.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 shutdown_connections 2024-04-26T22:27:53.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.323+0000 7f8a30a07640 1 -- 172.21.15.62:0/1887915121 wait complete. 2024-04-26T22:27:53.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T22:27:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.487+0000 7f4d5e859640 1 Processor -- start 2024-04-26T22:27:53.488 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.487+0000 7f4d5e859640 1 -- start start 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5e859640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5e859640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5e859640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d580d0960 con 0x7f4d580d5690 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5e859640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4d5815fa70 con 0x7f4d5815f690 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5e859640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4d580b5080 con 0x7f4d580d3380 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d57fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:34744/0 (socket says 172.21.15.62:34744) 2024-04-26T22:27:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d57fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:41214/0 (socket says 172.21.15.62:41214) 2024-04-26T22:27:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 -- 172.21.15.62:0/2837348916 learned_addr learned my addr 172.21.15.62:0/2837348916 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:53.490 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d5cdf6640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f4d580d5690 0x7f4d580d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:46848/0 (socket says 172.21.15.62:46848) 2024-04-26T22:27:53.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 -- 172.21.15.62:0/2837348916 >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 msgr2=0x7f4d580d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:53.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 --2- 172.21.15.62:0/2837348916 >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 -- 172.21.15.62:0/2837348916 >> v1:172.21.15.174:6789/0 conn(0x7f4d580d5690 legacy=0x7f4d580d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d577fe640 1 -- 172.21.15.62:0/2837348916 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d580d0c40 con 0x7f4d580d3380 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.488+0000 7f4d57fff640 1 --2- 172.21.15.62:0/2837348916 >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.489+0000 7f4d577fe640 1 --2- 172.21.15.62:0/2837348916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009870 tx=0x7f4d4c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b3cdc5ca48e3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.489+0000 7f4d56ffd640 1 -- 172.21.15.62:0/2837348916 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c037070 con 0x7f4d580d3380 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.489+0000 7f4d56ffd640 1 -- 172.21.15.62:0/2837348916 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4c032440 con 0x7f4d580d3380 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.489+0000 7f4d56ffd640 1 -- 172.21.15.62:0/2837348916 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c032810 con 0x7f4d580d3380 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 -- 172.21.15.62:0/2837348916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 msgr2=0x7f4d580d5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 --2- 172.21.15.62:0/2837348916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009870 tx=0x7f4d4c02a270 comp rx=0 tx=0).stop 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 -- 172.21.15.62:0/2837348916 shutdown_connections 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 --2- 172.21.15.62:0/2837348916 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d3380 0x7f4d580d5150 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 --2- 172.21.15.62:0/2837348916 >> v2:172.21.15.174:3300/0 conn(0x7f4d5815f690 0x7f4d580d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 -- 172.21.15.62:0/2837348916 >> 172.21.15.62:0/2837348916 conn(0x7f4d5808ebd0 msgr2=0x7f4d5809e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 -- 172.21.15.62:0/2837348916 shutdown_connections 2024-04-26T22:27:53.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.490+0000 7f4d5e859640 1 -- 172.21.15.62:0/2837348916 wait complete. 2024-04-26T22:27:53.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.491+0000 7f4d5e859640 1 Processor -- start 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.491+0000 7f4d5e859640 1 -- start start 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d580d3380 0x7f4d58158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d5690 0x7f4d580cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580d4010 con 0x7f4d5815f690 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580d1200 con 0x7f4d580d3380 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5e859640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4d580d4190 con 0x7f4d580d5690 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5cdf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5cdf6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37098/0 (socket says 172.21.15.62:37098) 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d5cdf6640 1 -- 172.21.15.62:0/591489490 learned_addr learned my addr 172.21.15.62:0/591489490 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d577fe640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d5690 0x7f4d580cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.492+0000 7f4d57fff640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d580d3380 0x7f4d58158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 -- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d5690 msgr2=0x7f4d580cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d5690 0x7f4d580cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 -- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d580d3380 msgr2=0x7f4d58158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d580d3380 0x7f4d58158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 -- 172.21.15.62:0/591489490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d581594d0 con 0x7f4d5815f690 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5cdf6640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c780 tx=0x7f4d5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50017070 con 0x7f4d5815f690 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5000a640 con 0x7f4d5815f690 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d580c8d70 con 0x7f4d5815f690 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5000a9f0 con 0x7f4d5815f690 2024-04-26T22:27:53.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.493+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d580c9330 con 0x7f4d5815f690 2024-04-26T22:27:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.494+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5806a360 con 0x7f4d5815f690 2024-04-26T22:27:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.495+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d5001a070 con 0x7f4d5815f690 2024-04-26T22:27:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.495+0000 7f4d54ff9640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 0x7f4d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.495+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f4d5008b1d0 con 0x7f4d5815f690 2024-04-26T22:27:53.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.495+0000 7f4d57fff640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 0x7f4d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:53.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.496+0000 7f4d57fff640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 0x7f4d4406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009560 tx=0x7f4d48009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:53.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.500+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d50058ed0 con 0x7f4d5815f690 2024-04-26T22:27:53.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:53.684+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d580b5ed0 con 0x7f4d5815f690 2024-04-26T22:27:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.325+0000 7f4d54ff9640 1 -- 172.21.15.62:0/591489490 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f4d5005d1d0 con 0x7f4d5815f690 2024-04-26T22:27:54.327 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.328+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 msgr2=0x7f4d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.328+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 0x7f4d4406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009560 tx=0x7f4d48009190 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.328+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 msgr2=0x7f4d580c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.328+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c780 tx=0x7f4d5000cc40 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 shutdown_connections 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4d4406c8f0 0x7f4d4406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d5815f690 0x7f4d580c9b30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4d580d5690 0x7f4d580cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 --2- 172.21.15.62:0/591489490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d580d3380 0x7f4d58158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 >> 172.21.15.62:0/591489490 conn(0x7f4d5808ebd0 msgr2=0x7f4d5809e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 shutdown_connections 2024-04-26T22:27:54.331 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.329+0000 7f4d5e859640 1 -- 172.21.15.62:0/591489490 wait complete. 2024-04-26T22:27:54.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T22:27:54.458 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.457+0000 7fecdf2b8640 1 Processor -- start 2024-04-26T22:27:54.459 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 -- start start 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecd80ce270 0x7fecd815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd80c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd80ced10 con 0x7fecd80cc920 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fecd80cf920 con 0x7fecd80ce270 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdf2b8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fecd80ceff0 con 0x7fecd80cc4a0 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.458+0000 7fecdd054640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecd80ce270 0x7fecd815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fecd80ce270 0x7fecd815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37114/0 (socket says 172.21.15.62:37114) 2024-04-26T22:27:54.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 -- 172.21.15.62:0/1510394841 learned_addr learned my addr 172.21.15.62:0/1510394841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:54.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdc853640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd80c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 -- 172.21.15.62:0/1510394841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 msgr2=0x7fecd80c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:54.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd80c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 -- 172.21.15.62:0/1510394841 >> v1:172.21.15.174:6789/0 conn(0x7fecd80cc920 legacy=0x7fecd80cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.459+0000 7fecdd054640 1 -- 172.21.15.62:0/1510394841 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd80cf2d0 con 0x7fecd80ce270 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdd054640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815ce60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009870 tx=0x7fecd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16963681e550a066 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7feccffff640 1 -- 172.21.15.62:0/1510394841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0037070 con 0x7fecd80ce270 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7feccffff640 1 -- 172.21.15.62:0/1510394841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd0032440 con 0x7fecd80ce270 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7feccffff640 1 -- 172.21.15.62:0/1510394841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd0032810 con 0x7fecd80ce270 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 -- 172.21.15.62:0/1510394841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 msgr2=0x7fecd815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.462 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815ce60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009870 tx=0x7fecd002a270 comp rx=0 tx=0).stop 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 -- 172.21.15.62:0/1510394841 shutdown_connections 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd80c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/1510394841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815ce60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 -- 172.21.15.62:0/1510394841 >> 172.21.15.62:0/1510394841 conn(0x7fecd80b2c60 msgr2=0x7fecd80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 -- 172.21.15.62:0/1510394841 shutdown_connections 2024-04-26T22:27:54.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.460+0000 7fecdf2b8640 1 -- 172.21.15.62:0/1510394841 wait complete. 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 Processor -- start 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 -- start start 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd81f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd80cc920 0x7fecd81f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fecd80ce9e0 con 0x7fecd80ce270 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecd80ce650 con 0x7fecd80cc920 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdf2b8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fecd806f3a0 con 0x7fecd80cc4a0 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.462+0000 7fecdd855640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37120/0 (socket says 172.21.15.62:37120) 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 -- 172.21.15.62:0/400041270 learned_addr learned my addr 172.21.15.62:0/400041270 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdc853640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd80cc920 0x7fecd81f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd054640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd81f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 -- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 msgr2=0x7fecd81f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd81f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 -- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd80cc920 msgr2=0x7fecd81f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd80cc920 0x7fecd81f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 -- 172.21.15.62:0/400041270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd81f5d70 con 0x7fecd80ce270 2024-04-26T22:27:54.466 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdd855640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c730 tx=0x7fecd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4016020 con 0x7fecd80ce270 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecd4006d50 con 0x7fecd80ce270 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd81f6050 con 0x7fecd80ce270 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4003bb0 con 0x7fecd80ce270 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.463+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd815a560 con 0x7fecd80ce270 2024-04-26T22:27:54.467 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.464+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd81f6220 con 0x7fecd80ce270 2024-04-26T22:27:54.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.465+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecd40050d0 con 0x7fecd80ce270 2024-04-26T22:27:54.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.465+0000 7feccdffb640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:54.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.465+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5568+0+0 (secure 0 0 0) 0x7fecd408b700 con 0x7fecd80ce270 2024-04-26T22:27:54.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.465+0000 7fecdd054640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:54.468 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.466+0000 7fecdd054640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 0x7fecbc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:54.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.470+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecd81f6220 con 0x7fecd80ce270 2024-04-26T22:27:54.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:54.655+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fecd806a360 con 0x7fecd80ce270 2024-04-26T22:27:55.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.326+0000 7feccdffb640 1 -- 172.21.15.62:0/400041270 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7fecd4020020 con 0x7fecd80ce270 2024-04-26T22:27:55.328 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.331+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 msgr2=0x7fecbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.331+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 0x7fecbc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fecd0034000 tx=0x7fecd002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.331+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 msgr2=0x7fecd815fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.331+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c730 tx=0x7fecd400cbf0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 shutdown_connections 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecbc06c8f0 0x7fecbc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fecd80ce270 0x7fecd815fcb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecd80cc920 0x7fecd81f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 --2- 172.21.15.62:0/400041270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecd80cc4a0 0x7fecd81f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.332+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 >> 172.21.15.62:0/400041270 conn(0x7fecd80b2c60 msgr2=0x7fecd80b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:55.334 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.333+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 shutdown_connections 2024-04-26T22:27:55.335 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.334+0000 7fecdf2b8640 1 -- 172.21.15.62:0/400041270 wait complete. 2024-04-26T22:27:55.353 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:27:55.353 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T22:27:55.354 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:27:55.354 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:27:55.354 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:27:55.354 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T22:27:55.354 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:27:55.354 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:27:55.354 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:27:55.355 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T22:27:55.355 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:27:55.355 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:27:55.355 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T22:27:55.355 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T22:27:55.355 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T22:27:55.355 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:27:55.365 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:27:55.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:27:55.366 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:27:55.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:27:55.367 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:27:55.367 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:27:55.368 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T22:27:55.368 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T22:27:55.407 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:27:55.408 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:55.409 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:27:55.409 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:55.410 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:27:55.410 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:55.410 INFO:tasks.ceph.mds.d:Started 2024-04-26T22:27:55.411 DEBUG:teuthology.parallel:result is None 2024-04-26T22:27:55.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T22:27:55.486 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2024-04-26T22:27:55.486 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2024-04-26T22:27:55.489 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-26T22:27:55.489 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-26T22:27:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.521+0000 7f16198f1640 1 Processor -- start 2024-04-26T22:27:55.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.521+0000 7f16198f1640 1 -- start start 2024-04-26T22:27:55.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16198f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f1614168150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16198f1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16140cf740 con 0x7f16140b1f90 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16198f1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16140b2400 con 0x7f1614160a00 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16198f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f16140d07e0 con 0x7f16140b3c90 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16127fc640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f16140b1f90 0x7f16140d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35954/0 (socket says 172.21.15.62:35954) 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16127fc640 1 -- 172.21.15.62:0/228235356 learned_addr learned my addr 172.21.15.62:0/228235356 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16137fe640 1 --2- 172.21.15.62:0/228235356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f1614168150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069367459 0 0) 0x7f16140cf740 con 0x7f16140b1f90 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15fc003520 con 0x7f16140b1f90 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299734491 0 0) 0x7f16140b2400 con 0x7f1614160a00 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16140cf740 con 0x7f1614160a00 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16137fe640 1 -- 172.21.15.62:0/228235356 >> v1:172.21.15.174:6789/0 conn(0x7f1614160a00 legacy=0x7f16140cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16137fe640 1 -- 172.21.15.62:0/228235356 >> v1:172.21.15.62:6789/0 conn(0x7f16140b1f90 legacy=0x7f16140d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.522+0000 7f16137fe640 1 -- 172.21.15.62:0/228235356 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16140cf080 con 0x7f16140b3c90 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.523+0000 7f16137fe640 1 --2- 172.21.15.62:0/228235356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f1614168150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f160c009dc0 tx=0x7f160c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a897541a2be1ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.523+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c007740 con 0x7f16140b3c90 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.523+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c0078e0 con 0x7f16140b3c90 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.523+0000 7f1611ffb640 1 -- 172.21.15.62:0/228235356 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c007c90 con 0x7f16140b3c90 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 -- 172.21.15.62:0/228235356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 msgr2=0x7f1614168150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 --2- 172.21.15.62:0/228235356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f1614168150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f160c009dc0 tx=0x7f160c02a760 comp rx=0 tx=0).stop 2024-04-26T22:27:55.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 -- 172.21.15.62:0/228235356 shutdown_connections 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 --2- 172.21.15.62:0/228235356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f1614168150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 -- 172.21.15.62:0/228235356 >> 172.21.15.62:0/228235356 conn(0x7f161409ba50 msgr2=0x7f161409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 -- 172.21.15.62:0/228235356 shutdown_connections 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.524+0000 7f16198f1640 1 -- 172.21.15.62:0/228235356 wait complete. 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.525+0000 7f16198f1640 1 Processor -- start 2024-04-26T22:27:55.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.525+0000 7f16198f1640 1 -- start start 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f16140b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1614160a00 0x7f16140b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f16140ceda0 con 0x7f16140b1f90 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16140ceac0 con 0x7f1614160a00 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16198f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1614160de0 con 0x7f16140b3c90 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f1612ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f1612ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37172/0 (socket says 172.21.15.62:37172) 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f1612ffd640 1 -- 172.21.15.62:0/4189896910 learned_addr learned my addr 172.21.15.62:0/4189896910 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16127fc640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f16140b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:55.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.526+0000 7f16137fe640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1614160a00 0x7f16140b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 -- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 msgr2=0x7f16140b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f16140b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 -- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1614160a00 msgr2=0x7f16140b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1614160a00 0x7f16140b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 -- 172.21.15.62:0/4189896910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16140b7270 con 0x7f16140b1f90 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f16127fc640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f16140b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f1612ffd640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f160400c3d0 tx=0x7f160400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604017070 con 0x7f16140b1f90 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16040052f0 con 0x7f16140b1f90 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16140b7550 con 0x7f16140b1f90 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604003830 con 0x7f16140b1f90 2024-04-26T22:27:55.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.527+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16140b7b10 con 0x7f16140b1f90 2024-04-26T22:27:55.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.528+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161406a360 con 0x7f16140b1f90 2024-04-26T22:27:55.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.529+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f160401a070 con 0x7f16140b1f90 2024-04-26T22:27:55.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.529+0000 7f15fbfff640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 0x7f160006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:55.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.529+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f160408aac0 con 0x7f16140b1f90 2024-04-26T22:27:55.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.529+0000 7f16127fc640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 0x7f160006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:55.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.530+0000 7f16127fc640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 0x7f160006ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1608007fd0 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:55.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.533+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1604059280 con 0x7f16140b1f90 2024-04-26T22:27:55.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:55.738+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f16140d25b0 con 0x7f16140b1f90 2024-04-26T22:27:56.345 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.344+0000 7f15fbfff640 1 -- 172.21.15.62:0/4189896910 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f1604058b80 con 0x7f16140b1f90 2024-04-26T22:27:56.345 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.346+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 msgr2=0x7f160006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.346+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 0x7f160006ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1608007fd0 tx=0x7f1608009040 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.346+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 msgr2=0x7f16140b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.346+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f160400c3d0 tx=0x7f160400c890 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 shutdown_connections 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f160006c920 0x7f160006ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1614160a00 0x7f16140b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f16140b3c90 0x7f16140b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 --2- 172.21.15.62:0/4189896910 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f16140b1f90 0x7f16140b4a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.348 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 >> 172.21.15.62:0/4189896910 conn(0x7f161409ba50 msgr2=0x7f161409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:56.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 shutdown_connections 2024-04-26T22:27:56.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.347+0000 7f16198f1640 1 -- 172.21.15.62:0/4189896910 wait complete. 2024-04-26T22:27:56.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T22:27:56.469 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.462+0000 7f7e1a719640 1 Processor -- start 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.462+0000 7f7e1a719640 1 -- start start 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e1a719640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e1406c240 0x7f7e1406fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e1a719640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e140b59b0 con 0x7f7e140b1480 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e1a719640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e14070350 con 0x7f7e1415f4f0 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e1a719640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f7e140b5c70 con 0x7f7e1406c240 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e13fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7e1406c240 0x7f7e1406fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e18cb6640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7e140b1480 0x7f7e140ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35966/0 (socket says 172.21.15.62:35966) 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e18cb6640 1 -- 172.21.15.62:0/3434417366 learned_addr learned my addr 172.21.15.62:0/3434417366 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e13fff640 1 -- 172.21.15.62:0/3434417366 >> v1:172.21.15.174:6789/0 conn(0x7f7e1415f4f0 legacy=0x7f7e1406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e13fff640 1 -- 172.21.15.62:0/3434417366 >> v1:172.21.15.62:6789/0 conn(0x7f7e140b1480 legacy=0x7f7e140ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e13fff640 1 -- 172.21.15.62:0/3434417366 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140b5f50 con 0x7f7e1406c240 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.463+0000 7f7e12ffd640 1 -- 172.21.15.62:0/3434417366 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566827880 0 0) 0x7f7e140b59b0 con 0x7f7e140b1480 2024-04-26T22:27:56.470 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e13fff640 1 --2- 172.21.15.62:0/3434417366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1406fd70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e080097a0 tx=0x7f7e0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d4f9bc2363282 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e12ffd640 1 -- 172.21.15.62:0/3434417366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08037070 con 0x7f7e1406c240 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e12ffd640 1 -- 172.21.15.62:0/3434417366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e08032440 con 0x7f7e1406c240 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e12ffd640 1 -- 172.21.15.62:0/3434417366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08032810 con 0x7f7e1406c240 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e1a719640 1 -- 172.21.15.62:0/3434417366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 msgr2=0x7f7e1406fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.464+0000 7f7e1a719640 1 --2- 172.21.15.62:0/3434417366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1406fd70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7e080097a0 tx=0x7f7e0802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.465+0000 7f7e1a719640 1 -- 172.21.15.62:0/3434417366 shutdown_connections 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.465+0000 7f7e1a719640 1 --2- 172.21.15.62:0/3434417366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1406fd70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.465+0000 7f7e1a719640 1 -- 172.21.15.62:0/3434417366 >> 172.21.15.62:0/3434417366 conn(0x7f7e140b3b70 msgr2=0x7f7e140b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.465+0000 7f7e1a719640 1 -- 172.21.15.62:0/3434417366 shutdown_connections 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.465+0000 7f7e1a719640 1 -- 172.21.15.62:0/3434417366 wait complete. 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.466+0000 7f7e1a719640 1 Processor -- start 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.466+0000 7f7e1a719640 1 -- start start 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.471 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 0x7f7e140d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 0x7f7e140d2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140b6750 con 0x7f7e1406c240 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e140b0fc0 con 0x7f7e1415f4f0 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e1a719640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7e140b68d0 con 0x7f7e140b1480 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37182/0 (socket says 172.21.15.62:37182) 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 -- 172.21.15.62:0/213481568 learned_addr learned my addr 172.21.15.62:0/213481568 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e18cb6640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 0x7f7e140d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e137fe640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 0x7f7e140d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 -- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 msgr2=0x7f7e140d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 0x7f7e140d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 -- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 msgr2=0x7f7e140d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 0x7f7e140d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.472 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 -- 172.21.15.62:0/213481568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e140cefd0 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e137fe640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 0x7f7e140d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e18cb6640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 0x7f7e140d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e13fff640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08041070 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.467+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e0803c930 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.468+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e140cf2b0 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.468+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08037040 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.468+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e140cf870 con 0x7f7e1406c240 2024-04-26T22:27:56.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.468+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e1406a360 con 0x7f7e1406c240 2024-04-26T22:27:56.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.469+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e08044070 con 0x7f7e1406c240 2024-04-26T22:27:56.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.469+0000 7f7e10ff9640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.470+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f7e080ac310 con 0x7f7e1406c240 2024-04-26T22:27:56.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.470+0000 7f7e137fe640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.474 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.470+0000 7f7e137fe640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 0x7f7e0006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e14069690 tx=0x7f7e04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.474+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e0807ab20 con 0x7f7e1406c240 2024-04-26T22:27:56.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.733+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7e140af6e0 con 0x7f7e1406c240 2024-04-26T22:27:56.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.733+0000 7f7e10ff9640 1 -- 172.21.15.62:0/213481568 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f7e0807a420 con 0x7f7e1406c240 2024-04-26T22:27:56.734 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:56.734 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmWg+9NRAAdnpMVpgAaGZ21j6W0Tbs+w==", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-26T22:27:56.735 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmwQbLBRAAuP30XZDLsJaBm6BNHTqFCA==", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm/wIqARAA55sdB593n/Skd4uA1syuSg==", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:27:56.736 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm0jNbEBAAudci4vHH/MXyeG7tCWL1Nw==", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-26T22:27:56.737 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQALKixmMfTWLxAAg/vI2T8zb3xX7r3ZgdrTFg==", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAOKixmfyK2LhAAHbPw2zCa8HuUWqsXiFPkPQ==", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.738 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQARKixmSo+dLBAAiPoVs57GokukpcqH47rfAw==", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUKixmXiJYKxAAo5iZwxDjIP6rn0cRysJxfg==", 2024-04-26T22:27:56.739 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAaKixmVSbbMBAA4oChxHAndPdjzYm3K59Acg==", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.740 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAdKixmTA5LLxAAtirJ58cd12oGcGYZ4W4eQg==", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAgKixm3+KjLRAAjf0ELmtEHU5hdTojxCPJDw==", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.741 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAjKixmm0jnKxAA7IYEt06xhis1drk5h0zH2w==", 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-26T22:27:56.742 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ==", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm20S/HRAAlRqwTgIuwG+sjvnjS87R1g==", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:27:56.743 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmZOyjGhAAxeZlr2qggbHBj9ZjmCsOqA==", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-26T22:27:56.744 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmfI08IxAAbwPyTVFRY5J5Vw08V31Ocw==", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmlpqGGhAAVIn6IHaa2aYjviVKuQEqFQ==", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:27:56.745 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mds", 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixmVhMiHxAAR0Cqf0+MSSIefcnIPQH9xw==", 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixmqBwiHxAAYAH0IaaEBoHpo6Hv62eihQ==", 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.746 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-osd", 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixmHSQiHxAABpiTCxLIwn57aLH8+EpVGw==", 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixmWSsiHxAAyCKE75qH1mG0g1J0p71piA==", 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T22:27:56.747 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixmEDMiHxAABP8U8s2l5Xnkxq+ULgs4Mw==", 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQApKixm7ToiHxAAojs+zkxti0BVEiMihu+Y/A==", 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.748 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmgseXMhAAtwa5QiDplGCQk5DQlFHHvg==", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixm0y23LhAA5KnsIydYZchjnmyUwLWl1w==", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:27:56.749 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-26T22:27:56.750 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:27:56.750 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.750 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:27:56.750 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-26T22:27:56.750 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-26T22:27:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 msgr2=0x7f7e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 0x7f7e0006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e14069690 tx=0x7f7e04008040 comp rx=0 tx=0).stop 2024-04-26T22:27:56.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 msgr2=0x7f7e1414c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000c00 tx=0x7f7e080046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 shutdown_connections 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7e0006c8f0 0x7f7e0006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e1415f4f0 0x7f7e140d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7e140b1480 0x7f7e140d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 --2- 172.21.15.62:0/213481568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7e1406c240 0x7f7e1414c8a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 >> 172.21.15.62:0/213481568 conn(0x7f7e140b3b70 msgr2=0x7f7e1406f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 shutdown_connections 2024-04-26T22:27:56.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.735+0000 7f7e1a719640 1 -- 172.21.15.62:0/213481568 wait complete. 2024-04-26T22:27:56.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T22:27:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.853+0000 7fcafbd19640 1 Processor -- start 2024-04-26T22:27:56.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.853+0000 7fcafbd19640 1 -- start start 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcafbd19640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcaf4162e40 0x7fcaf40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcafbd19640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcaf40b2050 0x7fcaf40cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcafbd19640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf40d0150 con 0x7fcaf40cfd70 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcafbd19640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf4163220 con 0x7fcaf4162e40 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcafbd19640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf40d0eb0 con 0x7fcaf40b2050 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcaf9ab5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcaf4162e40 0x7fcaf40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcaf9ab5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fcaf4162e40 0x7fcaf40d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37188/0 (socket says 172.21.15.62:37188) 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.854+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/90991935 learned_addr learned my addr 172.21.15.62:0/90991935 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/90991935 >> v2:172.21.15.174:3300/0 conn(0x7fcaf40b2050 msgr2=0x7fcaf40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf92b4640 1 --2- 172.21.15.62:0/90991935 >> v2:172.21.15.174:3300/0 conn(0x7fcaf40b2050 0x7fcaf40cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf9ab5640 1 --2- 172.21.15.62:0/90991935 >> v2:172.21.15.174:3300/0 conn(0x7fcaf40b2050 0x7fcaf40cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/90991935 >> v1:172.21.15.174:6789/0 conn(0x7fcaf40cfd70 legacy=0x7fcaf40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:56.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/90991935 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40b1d70 con 0x7fcaf4162e40 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.855+0000 7fcaf9ab5640 1 --2- 172.21.15.62:0/90991935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4162e40 0x7fcaf40d1340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4075e20 tx=0x7fcaf002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47c64a8f37aede3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcaf8ab3640 1 -- 172.21.15.62:0/90991935 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0037070 con 0x7fcaf4162e40 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcaf8ab3640 1 -- 172.21.15.62:0/90991935 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0032440 con 0x7fcaf4162e40 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcaf8ab3640 1 -- 172.21.15.62:0/90991935 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0032810 con 0x7fcaf4162e40 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 -- 172.21.15.62:0/90991935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4162e40 msgr2=0x7fcaf40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 --2- 172.21.15.62:0/90991935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4162e40 0x7fcaf40d1340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4075e20 tx=0x7fcaf002a160 comp rx=0 tx=0).stop 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 -- 172.21.15.62:0/90991935 shutdown_connections 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 --2- 172.21.15.62:0/90991935 >> v2:172.21.15.174:3300/0 conn(0x7fcaf40b2050 0x7fcaf40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 --2- 172.21.15.62:0/90991935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf4162e40 0x7fcaf40d1340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 -- 172.21.15.62:0/90991935 >> 172.21.15.62:0/90991935 conn(0x7fcaf409ba50 msgr2=0x7fcaf409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 -- 172.21.15.62:0/90991935 shutdown_connections 2024-04-26T22:27:56.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.856+0000 7fcafbd19640 1 -- 172.21.15.62:0/90991935 wait complete. 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.857+0000 7fcafbd19640 1 Processor -- start 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.857+0000 7fcafbd19640 1 -- start start 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaf40cfd70 0x7fcaf40b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 0x7fcaf41611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d1f20 con 0x7fcaf40b2050 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d1c40 con 0x7fcaf40cfd70 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafbd19640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf40b24e0 con 0x7fcaf4162e40 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcaf9ab5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcaf9ab5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37198/0 (socket says 172.21.15.62:37198) 2024-04-26T22:27:56.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/2800255191 learned_addr learned my addr 172.21.15.62:0/2800255191 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcaf92b4640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaf40cfd70 0x7fcaf40b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.858+0000 7fcafa2b6640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 0x7fcaf41611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 msgr2=0x7fcaf41611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 0x7fcaf41611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaf40cfd70 msgr2=0x7fcaf40b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaf40cfd70 0x7fcaf40b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 -- 172.21.15.62:0/2800255191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf4150a00 con 0x7fcaf40b2050 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcafa2b6640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 0x7fcaf41611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcaf9ab5640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002a1a0 tx=0x7fcaf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0037040 con 0x7fcaf40b2050 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf0042070 con 0x7fcaf40b2050 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf4150d10 con 0x7fcaf40b2050 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0030bb0 con 0x7fcaf40b2050 2024-04-26T22:27:56.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.859+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41506c0 con 0x7fcaf40b2050 2024-04-26T22:27:56.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.860+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf406a360 con 0x7fcaf40b2050 2024-04-26T22:27:56.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.861+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaf0047460 con 0x7fcaf40b2050 2024-04-26T22:27:56.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.861+0000 7fcae67fc640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:56.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.861+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fcaf00ac470 con 0x7fcaf40b2050 2024-04-26T22:27:56.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.861+0000 7fcaf92b4640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:56.864 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.862+0000 7fcaf92b4640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40c9aa0 tx=0x7fcae8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:56.866 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:56.865+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcaf4150d10 con 0x7fcaf40b2050 2024-04-26T22:27:57.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.126+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fcaf4069630 con 0x7fcaf40b2050 2024-04-26T22:27:57.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.128+0000 7fcae67fc640 1 -- 172.21.15.62:0/2800255191 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fcaf007abf0 con 0x7fcaf40b2050 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 msgr2=0x7fcad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 0x7fcad806edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40c9aa0 tx=0x7fcae8008040 comp rx=0 tx=0).stop 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 msgr2=0x7fcaf4150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002a1a0 tx=0x7fcaf00046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 shutdown_connections 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fcad806c8f0 0x7fcad806edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcaf4162e40 0x7fcaf41611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaf40cfd70 0x7fcaf40b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 --2- 172.21.15.62:0/2800255191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcaf40b2050 0x7fcaf4150180 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 >> 172.21.15.62:0/2800255191 conn(0x7fcaf409ba50 msgr2=0x7fcaf406e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 shutdown_connections 2024-04-26T22:27:57.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.130+0000 7fcafbd19640 1 -- 172.21.15.62:0/2800255191 wait complete. 2024-04-26T22:27:57.143 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T22:27:57.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 Processor -- start 2024-04-26T22:27:57.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 -- start start 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb14162e40 0x7feb140d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7feb140cff40 0x7feb140ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb140b1d70 con 0x7feb140b2050 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7feb14163220 con 0x7feb14162e40 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.243+0000 7feb1ad67640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7feb140d0eb0 con 0x7feb140cff40 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb14162e40 0x7feb140d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7feb14162e40 0x7feb140d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37200/0 (socket says 172.21.15.62:37200) 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 -- 172.21.15.62:0/3410972225 learned_addr learned my addr 172.21.15.62:0/3410972225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 -- 172.21.15.62:0/3410972225 >> v2:172.21.15.174:3300/0 conn(0x7feb140cff40 msgr2=0x7feb140ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 --2- 172.21.15.62:0/3410972225 >> v2:172.21.15.174:3300/0 conn(0x7feb140cff40 0x7feb140ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.247 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 -- 172.21.15.62:0/3410972225 >> v1:172.21.15.174:6789/0 conn(0x7feb140b2050 legacy=0x7feb140cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 -- 172.21.15.62:0/3410972225 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb140d1960 con 0x7feb14162e40 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.244+0000 7feb18b03640 1 --2- 172.21.15.62:0/3410972225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb14162e40 0x7feb140d1340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feb14073470 tx=0x7feb0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f237edb4a6722e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb137fe640 1 -- 172.21.15.62:0/3410972225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c037070 con 0x7feb14162e40 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb137fe640 1 -- 172.21.15.62:0/3410972225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c032440 con 0x7feb14162e40 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb137fe640 1 -- 172.21.15.62:0/3410972225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c032810 con 0x7feb14162e40 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 -- 172.21.15.62:0/3410972225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb14162e40 msgr2=0x7feb140d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 --2- 172.21.15.62:0/3410972225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb14162e40 0x7feb140d1340 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feb14073470 tx=0x7feb0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 -- 172.21.15.62:0/3410972225 shutdown_connections 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 --2- 172.21.15.62:0/3410972225 >> v2:172.21.15.174:3300/0 conn(0x7feb140cff40 0x7feb140ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 --2- 172.21.15.62:0/3410972225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb14162e40 0x7feb140d1340 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 -- 172.21.15.62:0/3410972225 >> 172.21.15.62:0/3410972225 conn(0x7feb1409ba50 msgr2=0x7feb1409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 -- 172.21.15.62:0/3410972225 shutdown_connections 2024-04-26T22:27:57.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.245+0000 7feb1ad67640 1 -- 172.21.15.62:0/3410972225 wait complete. 2024-04-26T22:27:57.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.246+0000 7feb1ad67640 1 Processor -- start 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.246+0000 7feb1ad67640 1 -- start start 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb140cff40 0x7feb1414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 0x7feb140b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb140b24e0 con 0x7feb140b2050 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb140d1f20 con 0x7feb140cff40 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb1ad67640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feb1406ea40 con 0x7feb14162e40 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb18b03640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb18b03640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37216/0 (socket says 172.21.15.62:37216) 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.247+0000 7feb18b03640 1 -- 172.21.15.62:0/100959830 learned_addr learned my addr 172.21.15.62:0/100959830 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb13fff640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb140cff40 0x7feb1414a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb19304640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 0x7feb140b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 -- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 msgr2=0x7feb140b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 0x7feb140b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 -- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb140cff40 msgr2=0x7feb1414a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb140cff40 0x7feb1414a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 -- 172.21.15.62:0/100959830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb140b7e00 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb19304640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 0x7feb140b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb18b03640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c041070 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c03c870 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb140b80e0 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c037040 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.248+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb14157060 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.249+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1406a360 con 0x7feb140b2050 2024-04-26T22:27:57.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.250+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb0c044070 con 0x7feb140b2050 2024-04-26T22:27:57.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.250+0000 7feb117fa640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 0x7feaf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.250+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7feb0c0ac570 con 0x7feb140b2050 2024-04-26T22:27:57.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.251+0000 7feb13fff640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 0x7feaf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.251+0000 7feb13fff640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 0x7feaf406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb140c9960 tx=0x7feb04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.255+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb0c07aea0 con 0x7feb140b2050 2024-04-26T22:27:57.517 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.516+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7feb14070d80 con 0x7feb140b2050 2024-04-26T22:27:57.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.518+0000 7feb117fa640 1 -- 172.21.15.62:0/100959830 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7feb0c07a7a0 con 0x7feb140b2050 2024-04-26T22:27:57.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 msgr2=0x7feaf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 0x7feaf406edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feb140c9960 tx=0x7feb04009040 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 msgr2=0x7feb141621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 shutdown_connections 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feaf406c8f0 0x7feaf406edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb14162e40 0x7feb140b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb140cff40 0x7feb1414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 --2- 172.21.15.62:0/100959830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb140b2050 0x7feb141621b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 >> 172.21.15.62:0/100959830 conn(0x7feb1409ba50 msgr2=0x7feb140b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 shutdown_connections 2024-04-26T22:27:57.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.520+0000 7feb1ad67640 1 -- 172.21.15.62:0/100959830 wait complete. 2024-04-26T22:27:57.532 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T22:27:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.632+0000 7ff3b5e15640 1 Processor -- start 2024-04-26T22:27:57.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.632+0000 7ff3b5e15640 1 -- start start 2024-04-26T22:27:57.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3b5e15640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3b00d22e0 0x7ff3b00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3b5e15640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff3b015f760 0x7ff3b00b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3b5e15640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b00d3de0 con 0x7ff3b015f380 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3b5e15640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b00d43a0 con 0x7ff3b00d22e0 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3b5e15640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b00d40c0 con 0x7ff3b015f760 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3aeffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3b00d22e0 0x7ff3b00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3aeffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff3b00d22e0 0x7ff3b00b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37220/0 (socket says 172.21.15.62:37220) 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.633+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2713303633 learned_addr learned my addr 172.21.15.62:0/2713303633 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3affff640 1 --2- 172.21.15.62:0/2713303633 >> v2:172.21.15.174:3300/0 conn(0x7ff3b015f760 0x7ff3b00b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2713303633 >> v2:172.21.15.174:3300/0 conn(0x7ff3b015f760 msgr2=0x7ff3b00b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3aeffd640 1 --2- 172.21.15.62:0/2713303633 >> v2:172.21.15.174:3300/0 conn(0x7ff3b015f760 0x7ff3b00b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.637 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2713303633 >> v1:172.21.15.174:6789/0 conn(0x7ff3b015f380 legacy=0x7ff3b00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2713303633 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b00b8280 con 0x7ff3b00d22e0 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3aeffd640 1 --2- 172.21.15.62:0/2713303633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00d22e0 0x7ff3b00b8a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a445b173260cee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.634+0000 7ff3ae7fc640 1 -- 172.21.15.62:0/2713303633 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4037070 con 0x7ff3b00d22e0 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3ae7fc640 1 -- 172.21.15.62:0/2713303633 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a4032640 con 0x7ff3b00d22e0 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3ae7fc640 1 -- 172.21.15.62:0/2713303633 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4032a10 con 0x7ff3b00d22e0 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2713303633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00d22e0 msgr2=0x7ff3b00b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2713303633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00d22e0 0x7ff3b00b8a80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a402a300 comp rx=0 tx=0).stop 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2713303633 shutdown_connections 2024-04-26T22:27:57.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2713303633 >> v2:172.21.15.174:3300/0 conn(0x7ff3b015f760 0x7ff3b00b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2713303633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b00d22e0 0x7ff3b00b8a80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2713303633 >> 172.21.15.62:0/2713303633 conn(0x7ff3b009e560 msgr2=0x7ff3b00d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:57.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2713303633 shutdown_connections 2024-04-26T22:27:57.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.635+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2713303633 wait complete. 2024-04-26T22:27:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.636+0000 7ff3b5e15640 1 Processor -- start 2024-04-26T22:27:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.636+0000 7ff3b5e15640 1 -- start start 2024-04-26T22:27:57.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3b00d22e0 0x7ff3b0160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3b015f760 0x7ff3b0162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b00d1280 con 0x7ff3b015f380 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b00d1100 con 0x7ff3b00d22e0 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3b5e15640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b00d5b50 con 0x7ff3b015f760 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3aeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37234/0 (socket says 172.21.15.62:37234) 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2678867055 learned_addr learned my addr 172.21.15.62:0/2678867055 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3af7fe640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3b00d22e0 0x7ff3b0160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.637+0000 7ff3affff640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3b015f760 0x7ff3b0162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3b015f760 msgr2=0x7ff3b0162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3b015f760 0x7ff3b0162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3b00d22e0 msgr2=0x7ff3b0160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3b00d22e0 0x7ff3b0160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 -- 172.21.15.62:0/2678867055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b0145510 con 0x7ff3b015f380 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3aeffd640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4037040 con 0x7ff3b015f380 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a4004030 con 0x7ff3b015f380 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b01627a0 con 0x7ff3b015f380 2024-04-26T22:27:57.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4004420 con 0x7ff3b015f380 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.638+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b0154cb0 con 0x7ff3b015f380 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.639+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b006a360 con 0x7ff3b015f380 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.639+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3a40306f0 con 0x7ff3b015f380 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.640+0000 7ff397fff640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 0x7ff39c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.640+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7ff3a40ac040 con 0x7ff3b015f380 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.640+0000 7ff3af7fe640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 0x7ff39c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:57.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.641+0000 7ff3af7fe640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 0x7ff39c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0004f60 tx=0x7ff3a0007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:57.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.644+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3a407a0e0 con 0x7ff3b015f380 2024-04-26T22:27:57.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.902+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7ff3b015df30 con 0x7ff3b015f380 2024-04-26T22:27:57.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.904+0000 7ff397fff640 1 -- 172.21.15.62:0/2678867055 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7ff3a407e3e0 con 0x7ff3b015f380 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 msgr2=0x7ff39c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 0x7ff39c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0004f60 tx=0x7ff3a0007e30 comp rx=0 tx=0).stop 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 msgr2=0x7ff3b0161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009870 tx=0x7ff3a40046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 shutdown_connections 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff39c06c8f0 0x7ff39c06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3b015f760 0x7ff3b0162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff3b015f380 0x7ff3b0161be0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 --2- 172.21.15.62:0/2678867055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3b00d22e0 0x7ff3b0160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:57.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 >> 172.21.15.62:0/2678867055 conn(0x7ff3b009e560 msgr2=0x7ff3b00b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:57.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 shutdown_connections 2024-04-26T22:27:57.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:57.906+0000 7ff3b5e15640 1 -- 172.21.15.62:0/2678867055 wait complete. 2024-04-26T22:27:57.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T22:27:58.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.018+0000 7fca975b4640 1 Processor -- start 2024-04-26T22:27:58.019 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.018+0000 7fca975b4640 1 -- start start 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca975b4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fca9015fc10 0x7fca901600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca975b4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca900ce9b0 0x7fca900cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca975b4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca90072e30 con 0x7fca90159a90 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca975b4640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fca90073110 con 0x7fca900ce9b0 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca975b4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fca900733f0 con 0x7fca9015fc10 2024-04-26T22:27:58.021 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95350640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fca90159a90 0x7fca9006fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35978/0 (socket says 172.21.15.62:35978) 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95350640 1 -- 172.21.15.62:0/439963614 learned_addr learned my addr 172.21.15.62:0/439963614 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95b51640 1 --2- 172.21.15.62:0/439963614 >> v2:172.21.15.62:3300/0 conn(0x7fca900ce9b0 0x7fca900cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca87fff640 1 -- 172.21.15.62:0/439963614 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891099342 0 0) 0x7fca90072e30 con 0x7fca90159a90 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca87fff640 1 -- 172.21.15.62:0/439963614 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca7c003520 con 0x7fca90159a90 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca94b4f640 1 --2- 172.21.15.62:0/439963614 >> v2:172.21.15.174:3300/0 conn(0x7fca9015fc10 0x7fca901600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95b51640 1 -- 172.21.15.62:0/439963614 >> v2:172.21.15.174:3300/0 conn(0x7fca9015fc10 msgr2=0x7fca901600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95b51640 1 --2- 172.21.15.62:0/439963614 >> v2:172.21.15.174:3300/0 conn(0x7fca9015fc10 0x7fca901600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95b51640 1 -- 172.21.15.62:0/439963614 >> v1:172.21.15.62:6789/0 conn(0x7fca90159a90 legacy=0x7fca9006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.019+0000 7fca95b51640 1 -- 172.21.15.62:0/439963614 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca900736d0 con 0x7fca900ce9b0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.020+0000 7fca95b51640 1 --2- 172.21.15.62:0/439963614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca900ce9b0 0x7fca900cee20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca9015d630 tx=0x7fca8802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6aa3c1d29fbe5b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.020+0000 7fca87fff640 1 -- 172.21.15.62:0/439963614 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88007740 con 0x7fca900ce9b0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.020+0000 7fca87fff640 1 -- 172.21.15.62:0/439963614 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca880078e0 con 0x7fca900ce9b0 2024-04-26T22:27:58.022 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.020+0000 7fca87fff640 1 -- 172.21.15.62:0/439963614 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88007cb0 con 0x7fca900ce9b0 2024-04-26T22:27:58.025 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 -- 172.21.15.62:0/439963614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca900ce9b0 msgr2=0x7fca900cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 --2- 172.21.15.62:0/439963614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca900ce9b0 0x7fca900cee20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca9015d630 tx=0x7fca8802a560 comp rx=0 tx=0).stop 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 -- 172.21.15.62:0/439963614 shutdown_connections 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 --2- 172.21.15.62:0/439963614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca900ce9b0 0x7fca900cee20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 --2- 172.21.15.62:0/439963614 >> v2:172.21.15.174:3300/0 conn(0x7fca9015fc10 0x7fca901600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 -- 172.21.15.62:0/439963614 >> 172.21.15.62:0/439963614 conn(0x7fca900d0d50 msgr2=0x7fca900d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 -- 172.21.15.62:0/439963614 shutdown_connections 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.021+0000 7fca975b4640 1 -- 172.21.15.62:0/439963614 wait complete. 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.022+0000 7fca975b4640 1 Processor -- start 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.022+0000 7fca975b4640 1 -- start start 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 0x7fca90144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 0x7fca9015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.026 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fca90159e70 con 0x7fca90159a90 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca90070300 con 0x7fca900ce9b0 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca975b4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fca90159ff0 con 0x7fca9015fc10 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37252/0 (socket says 172.21.15.62:37252) 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 -- 172.21.15.62:0/3242943696 learned_addr learned my addr 172.21.15.62:0/3242943696 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca95350640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 0x7fca90144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 -- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 msgr2=0x7fca9015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca95b51640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 0x7fca9015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 0x7fca9015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 -- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 msgr2=0x7fca90144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 0x7fca90144ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 -- 172.21.15.62:0/3242943696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca90169640 con 0x7fca90159a90 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca95350640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 0x7fca90144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca95b51640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 0x7fca9015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:58.027 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.023+0000 7fca94b4f640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d5f0 tx=0x7fca8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.024+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800125f0 con 0x7fca90159a90 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.024+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca80019070 con 0x7fca90159a90 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.024+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800129d0 con 0x7fca90159a90 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.024+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca900b4bd0 con 0x7fca90159a90 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.024+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca900b5190 con 0x7fca90159a90 2024-04-26T22:27:58.028 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.025+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9006a360 con 0x7fca90159a90 2024-04-26T22:27:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.025+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca80012b70 con 0x7fca90159a90 2024-04-26T22:27:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.026+0000 7fca85ffb640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 0x7fca7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.026+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fca8008bb70 con 0x7fca90159a90 2024-04-26T22:27:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.026+0000 7fca95350640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 0x7fca7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.029 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.026+0000 7fca95350640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 0x7fca7406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005880 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.031 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.030+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca8005a380 con 0x7fca90159a90 2024-04-26T22:27:58.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.289+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fca900d0730 con 0x7fca90159a90 2024-04-26T22:27:58.292 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.291+0000 7fca85ffb640 1 -- 172.21.15.62:0/3242943696 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fca80059c80 con 0x7fca90159a90 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 msgr2=0x7fca7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 0x7fca7406edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca8c005880 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 msgr2=0x7fca90143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d5f0 tx=0x7fca8000dab0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 shutdown_connections 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca7406c8f0 0x7fca7406edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca9015fc10 0x7fca9015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca90159a90 0x7fca90143100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 --2- 172.21.15.62:0/3242943696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca900ce9b0 0x7fca90144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 >> 172.21.15.62:0/3242943696 conn(0x7fca900d0d50 msgr2=0x7fca900b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 shutdown_connections 2024-04-26T22:27:58.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.293+0000 7fca975b4640 1 -- 172.21.15.62:0/3242943696 wait complete. 2024-04-26T22:27:58.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T22:27:58.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.406+0000 7fb871b38640 1 Processor -- start 2024-04-26T22:27:58.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.406+0000 7fb871b38640 1 -- start start 2024-04-26T22:27:58.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb871b38640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb86c0cff40 0x7fb86c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb871b38640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c0b1d70 con 0x7fb86c162e40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb871b38640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86c0d1960 con 0x7fb86c06ea40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb871b38640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb86c163220 con 0x7fb86c0cff40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86b7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb86c162e40 0x7fb86c0b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:35988/0 (socket says 172.21.15.62:35988) 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86b7fe640 1 -- 172.21.15.62:0/190231209 learned_addr learned my addr 172.21.15.62:0/190231209 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133282216 0 0) 0x7fb86c0b1d70 con 0x7fb86c162e40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86bfff640 1 --2- 172.21.15.62:0/190231209 >> v2:172.21.15.174:3300/0 conn(0x7fb86c0cff40 0x7fb86c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb854003520 con 0x7fb86c162e40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4232337858 0 0) 0x7fb854003520 con 0x7fb86c162e40 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 >> v2:172.21.15.174:3300/0 conn(0x7fb86c0cff40 msgr2=0x7fb86c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 --2- 172.21.15.62:0/190231209 >> v2:172.21.15.174:3300/0 conn(0x7fb86c0cff40 0x7fb86c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 >> v1:172.21.15.174:6789/0 conn(0x7fb86c06ea40 legacy=0x7fb86c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c0d1c40 con 0x7fb86c162e40 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fb85c001670 con 0x7fb86c162e40 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 >> v1:172.21.15.62:6789/0 conn(0x7fb86c162e40 legacy=0x7fb86c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8540039b0 0x7fb854003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.407+0000 7fb86a7fc640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb854004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86a7fc640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8540051c0 0x7fb85400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb85400c130 con 0x7fb854004800 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb85400c2e0 con 0x7fb8540051c0 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb85400c4e0 con 0x7fb8540039b0 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb854004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86b7fe640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8540039b0 0x7fb854003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86bfff640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8540051c0 0x7fb85400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 -- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8540039b0 msgr2=0x7fb854003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8540039b0 0x7fb854003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 -- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8540051c0 msgr2=0x7fb85400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8540051c0 0x7fb85400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86affd640 1 -- 172.21.15.62:0/190231209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854003520 con 0x7fb854004800 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.408+0000 7fb86bfff640 1 -- 172.21.15.62:0/190231209 reap_dead start 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb86affd640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb854004c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb860001750 tx=0x7fb854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257b73c381ed2ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860001370 con 0x7fb854004800 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8600052d0 con 0x7fb854004800 2024-04-26T22:27:58.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb86a7fc640 1 -- 172.21.15.62:0/190231209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86002abc0 con 0x7fb854004800 2024-04-26T22:27:58.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 -- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 msgr2=0x7fb854004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb854004c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb860001750 tx=0x7fb854005830 comp rx=0 tx=0).stop 2024-04-26T22:27:58.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 -- 172.21.15.62:0/190231209 shutdown_connections 2024-04-26T22:27:58.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 --2- 172.21.15.62:0/190231209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb854004c80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 -- 172.21.15.62:0/190231209 >> 172.21.15.62:0/190231209 conn(0x7fb86c09ba50 msgr2=0x7fb86c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 -- 172.21.15.62:0/190231209 shutdown_connections 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.409+0000 7fb871b38640 1 -- 172.21.15.62:0/190231209 wait complete. 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.410+0000 7fb871b38640 1 Processor -- start 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 -- start start 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 0x7fb86c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 0x7fb86c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0d1340 con 0x7fb854004800 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c0d0eb0 con 0x7fb86c0afd50 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb871b38640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb86c0d14c0 con 0x7fb86c15cd90 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.411+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37264/0 (socket says 172.21.15.62:37264) 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 -- 172.21.15.62:0/2603588531 learned_addr learned my addr 172.21.15.62:0/2603588531 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86affd640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 0x7fb86c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86bfff640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 0x7fb86c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 -- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 msgr2=0x7fb86c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 0x7fb86c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 -- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 msgr2=0x7fb86c0b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 0x7fb86c0b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 -- 172.21.15.62:0/2603588531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c0c7c00 con 0x7fb854004800 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86affd640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 0x7fb86c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86bfff640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 0x7fb86c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb86b7fe640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb85c007320 tx=0x7fb85c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c01a070 con 0x7fb854004800 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c00f8e0 con 0x7fb854004800 2024-04-26T22:27:58.417 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00fcd0 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c0c7ea0 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.412+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c0c8460 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.413+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c06a360 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.414+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb85c01d070 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.414+0000 7fb853fff640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 0x7fb85806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.414+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fb85c08b070 con 0x7fb854004800 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.414+0000 7fb86affd640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 0x7fb85806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.415+0000 7fb86affd640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 0x7fb85806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb860032000 tx=0x7fb86002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.419+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb85c059880 con 0x7fb854004800 2024-04-26T22:27:58.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.673+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb86c075c50 con 0x7fb854004800 2024-04-26T22:27:58.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.675+0000 7fb853fff640 1 -- 172.21.15.62:0/2603588531 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fb85c059180 con 0x7fb854004800 2024-04-26T22:27:58.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 msgr2=0x7fb85806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 0x7fb85806ede0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb860032000 tx=0x7fb86002e000 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 msgr2=0x7fb86c1f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb85c007320 tx=0x7fb85c0077e0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 shutdown_connections 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb85806c920 0x7fb85806ede0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb86c15cd90 0x7fb86c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c0afd50 0x7fb86c0b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 --2- 172.21.15.62:0/2603588531 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb854004800 0x7fb86c1f48d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 >> 172.21.15.62:0/2603588531 conn(0x7fb86c09ba50 msgr2=0x7fb86c160cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 shutdown_connections 2024-04-26T22:27:58.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.677+0000 7fb871b38640 1 -- 172.21.15.62:0/2603588531 wait complete. 2024-04-26T22:27:58.689 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T22:27:58.790 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.789+0000 7f2c6a615640 1 Processor -- start 2024-04-26T22:27:58.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.789+0000 7f2c6a615640 1 -- start start 2024-04-26T22:27:58.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c6a615640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c640b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c6a615640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c640d3de0 con 0x7f2c6415f760 2024-04-26T22:27:58.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c6a615640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c640d40c0 con 0x7f2c6415f380 2024-04-26T22:27:58.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c6a615640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c640d43a0 con 0x7f2c640d22e0 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c68bb2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2c6415f760 0x7f2c640b8fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36000/0 (socket says 172.21.15.62:36000) 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c68bb2640 1 -- 172.21.15.62:0/258144189 learned_addr learned my addr 172.21.15.62:0/258144189 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c637fe640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c640b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3220003403 0 0) 0x7f2c640d3de0 con 0x7f2c6415f760 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.790+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c50003520 con 0x7f2c6415f760 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 795412128 0 0) 0x7f2c50003520 con 0x7f2c6415f760 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 msgr2=0x7f2c640b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c640b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.795 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 >> v1:172.21.15.174:6789/0 conn(0x7f2c6415f380 legacy=0x7f2c640d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c640b8280 con 0x7f2c6415f760 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f2c5c0019f0 con 0x7f2c6415f760 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 >> v1:172.21.15.62:6789/0 conn(0x7f2c6415f760 legacy=0x7f2c640b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c500039e0 0x7f2c50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c50004d40 0x7f2c500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640d43a0 con 0x7f2c50004d40 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5000ba20 con 0x7f2c500039e0 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5000bc20 con 0x7f2c640d22e0 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c50004d40 0x7f2c500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c637fe640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c63fff640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c500039e0 0x7f2c50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 -- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 msgr2=0x7f2c50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640d22e0 0x7f2c50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 -- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c500039e0 msgr2=0x7f2c50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c500039e0 0x7f2c50003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.791+0000 7f2c68bb2640 1 -- 172.21.15.62:0/258144189 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50003520 con 0x7f2c50004d40 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c50004d40 0x7f2c500051c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c6415fbd0 tx=0x7f2c50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf64e89e76372ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00a410 con 0x7f2c50004d40 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5c02f070 con 0x7f2c50004d40 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c62ffd640 1 -- 172.21.15.62:0/258144189 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c02bbf0 con 0x7f2c50004d40 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c6a615640 1 -- 172.21.15.62:0/258144189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c50004d40 msgr2=0x7f2c500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.797 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c6a615640 1 --2- 172.21.15.62:0/258144189 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c50004d40 0x7f2c500051c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2c6415fbd0 tx=0x7f2c50005830 comp rx=0 tx=0).stop 2024-04-26T22:27:58.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.792+0000 7f2c68bb2640 1 -- 172.21.15.62:0/258144189 reap_dead start 2024-04-26T22:27:58.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.793+0000 7f2c6a615640 1 -- 172.21.15.62:0/258144189 shutdown_connections 2024-04-26T22:27:58.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.793+0000 7f2c6a615640 1 -- 172.21.15.62:0/258144189 >> 172.21.15.62:0/258144189 conn(0x7f2c6409e560 msgr2=0x7f2c640d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:58.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.793+0000 7f2c6a615640 1 -- 172.21.15.62:0/258144189 shutdown_connections 2024-04-26T22:27:58.798 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.793+0000 7f2c6a615640 1 -- 172.21.15.62:0/258144189 wait complete. 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.794+0000 7f2c6a615640 1 Processor -- start 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.794+0000 7f2c6a615640 1 -- start start 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 0x7f2c641f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640c89b0 0x7f2c6415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640d1280 con 0x7f2c640cb8d0 2024-04-26T22:27:58.801 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c640d1100 con 0x7f2c50004d40 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c640d5b50 con 0x7f2c640c89b0 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37278/0 (socket says 172.21.15.62:37278) 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 -- 172.21.15.62:0/3272004965 learned_addr learned my addr 172.21.15.62:0/3272004965 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c637fe640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640c89b0 0x7f2c6415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c63fff640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 0x7f2c641f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 -- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640c89b0 msgr2=0x7f2c6415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640c89b0 0x7f2c6415b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 -- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 msgr2=0x7f2c641f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 0x7f2c641f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:58.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 -- 172.21.15.62:0/3272004965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c640cb130 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c63fff640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 0x7f2c641f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c68bb2640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f80 tx=0x7f2c640b8450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c001f50 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c5c0019e0 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c004da0 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6415bd60 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.795+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c641f5110 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.797+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6406a360 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.797+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c5c047060 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.797+0000 7f2c60ff9640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 0x7f2c4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.797+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f2c5c0ab200 con 0x7f2c640cb8d0 2024-04-26T22:27:58.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.797+0000 7f2c63fff640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 0x7f2c4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.798+0000 7f2c63fff640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 0x7f2c4406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:58.804 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:58.802+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c640cb130 con 0x7f2c640cb8d0 2024-04-26T22:27:59.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.060+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2c64076020 con 0x7f2c640cb8d0 2024-04-26T22:27:59.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.062+0000 7f2c60ff9640 1 -- 172.21.15.62:0/3272004965 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f2c5c0333d0 con 0x7f2c640cb8d0 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 msgr2=0x7f2c4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 0x7f2c4406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54009040 comp rx=0 tx=0).stop 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 msgr2=0x7f2c640cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000f80 tx=0x7f2c640b8450 comp rx=0 tx=0).stop 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 shutdown_connections 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c4406c8f0 0x7f2c4406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c640cb8d0 0x7f2c640cbd50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c640c89b0 0x7f2c6415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 --2- 172.21.15.62:0/3272004965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50004d40 0x7f2c641f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 >> 172.21.15.62:0/3272004965 conn(0x7f2c6409e560 msgr2=0x7f2c64161bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 shutdown_connections 2024-04-26T22:27:59.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.065+0000 7f2c6a615640 1 -- 172.21.15.62:0/3272004965 wait complete. 2024-04-26T22:27:59.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 Processor -- start 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 -- start start 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4ce406ea40 0x7f4ce40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ce40cf830 0x7f4ce40cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ce40d0180 con 0x7f4ce4162e40 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce4163220 con 0x7f4ce40cf830 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.177+0000 7f4cea425640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4ce40d0eb0 con 0x7f4ce406ea40 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce89c2640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f4ce40cf830 0x7f4ce40cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce3fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4ce4162e40 0x7f4ce40b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36012/0 (socket says 172.21.15.62:36012) 2024-04-26T22:27:59.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce3fff640 1 -- 172.21.15.62:0/3695663397 learned_addr learned my addr 172.21.15.62:0/3695663397 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce89c2640 1 -- 172.21.15.62:0/3695663397 >> v2:172.21.15.174:3300/0 conn(0x7f4ce406ea40 msgr2=0x7f4ce40d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce37fe640 1 --2- 172.21.15.62:0/3695663397 >> v2:172.21.15.174:3300/0 conn(0x7f4ce406ea40 0x7f4ce40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce89c2640 1 --2- 172.21.15.62:0/3695663397 >> v2:172.21.15.174:3300/0 conn(0x7f4ce406ea40 0x7f4ce40d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce89c2640 1 -- 172.21.15.62:0/3695663397 >> v1:172.21.15.62:6789/0 conn(0x7f4ce4162e40 legacy=0x7f4ce40b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce89c2640 1 -- 172.21.15.62:0/3695663397 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce40b1d70 con 0x7f4ce40cf830 2024-04-26T22:27:59.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.178+0000 7f4ce2ffd640 1 -- 172.21.15.62:0/3695663397 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2175984905 0 0) 0x7f4cd4001110 con 0x7f4ce4162e40 2024-04-26T22:27:59.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4ce89c2640 1 --2- 172.21.15.62:0/3695663397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce40cf830 0x7f4ce40cfc40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4072bb0 tx=0x7f4cdc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf79aa209b5603b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4ce2ffd640 1 -- 172.21.15.62:0/3695663397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007740 con 0x7f4ce40cf830 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4ce2ffd640 1 -- 172.21.15.62:0/3695663397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cdc0078e0 con 0x7f4ce40cf830 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4ce2ffd640 1 -- 172.21.15.62:0/3695663397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007cb0 con 0x7f4ce40cf830 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 -- 172.21.15.62:0/3695663397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce40cf830 msgr2=0x7f4ce40cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 --2- 172.21.15.62:0/3695663397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce40cf830 0x7f4ce40cfc40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4072bb0 tx=0x7f4cdc02a5f0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 -- 172.21.15.62:0/3695663397 shutdown_connections 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 --2- 172.21.15.62:0/3695663397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce40cf830 0x7f4ce40cfc40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 --2- 172.21.15.62:0/3695663397 >> v2:172.21.15.174:3300/0 conn(0x7f4ce406ea40 0x7f4ce40d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 -- 172.21.15.62:0/3695663397 >> 172.21.15.62:0/3695663397 conn(0x7f4ce409ba50 msgr2=0x7f4ce409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 -- 172.21.15.62:0/3695663397 shutdown_connections 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.179+0000 7f4cea425640 1 -- 172.21.15.62:0/3695663397 wait complete. 2024-04-26T22:27:59.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.180+0000 7f4cea425640 1 Processor -- start 2024-04-26T22:27:59.183 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 -- start start 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 0x7f4ce4071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 0x7f4ce4071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40d1f20 con 0x7f4ce406ea40 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40d1c40 con 0x7f4ce40cf830 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4cea425640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40ce010 con 0x7f4ce4162e40 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4ce3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4ce3fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37308/0 (socket says 172.21.15.62:37308) 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.181+0000 7f4ce3fff640 1 -- 172.21.15.62:0/2497147443 learned_addr learned my addr 172.21.15.62:0/2497147443 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce89c2640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 0x7f4ce4071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce37fe640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 0x7f4ce4071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.185 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 -- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 msgr2=0x7f4ce4071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 0x7f4ce4071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 -- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 msgr2=0x7f4ce4071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 0x7f4ce4071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 -- 172.21.15.62:0/2497147443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce40cafd0 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce89c2640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 0x7f4ce4071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce37fe640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 0x7f4ce4071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce3fff640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c460 tx=0x7f4cd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4018070 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd4007500 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce40cb2b0 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4004a20 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.182+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce40cb870 con 0x7f4ce406ea40 2024-04-26T22:27:59.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.183+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce406a360 con 0x7f4ce406ea40 2024-04-26T22:27:59.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.183+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4cd4004bc0 con 0x7f4ce406ea40 2024-04-26T22:27:59.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.184+0000 7f4ce0ff9640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 0x7f4cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.184+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f4cd408ab00 con 0x7f4ce406ea40 2024-04-26T22:27:59.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.184+0000 7f4ce37fe640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 0x7f4cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.185+0000 7f4ce37fe640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 0x7f4cd006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ce415b890 tx=0x7f4cd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.188+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ce40d0180 con 0x7f4ce406ea40 2024-04-26T22:27:59.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.378+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ce41622a0 con 0x7f4ce406ea40 2024-04-26T22:27:59.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.379+0000 7f4ce0ff9640 1 -- 172.21.15.62:0/2497147443 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+12891 (secure 0 0 0) 0x7f4cd4059290 con 0x7f4ce406ea40 2024-04-26T22:27:59.380 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:27:59.380 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":27,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:27:56.354922+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 msgr2=0x7f4cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 0x7f4cd006edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ce415b890 tx=0x7f4cd8008040 comp rx=0 tx=0).stop 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 msgr2=0x7f4ce4160520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c460 tx=0x7f4cd400c920 comp rx=0 tx=0).stop 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 shutdown_connections 2024-04-26T22:27:59.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4cd006c8f0 0x7f4cd006edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ce4162e40 0x7f4ce4071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce40cf830 0x7f4ce4071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 --2- 172.21.15.62:0/2497147443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ce406ea40 0x7f4ce4160520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 >> 172.21.15.62:0/2497147443 conn(0x7f4ce409ba50 msgr2=0x7f4ce40ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 shutdown_connections 2024-04-26T22:27:59.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.381+0000 7f4cea425640 1 -- 172.21.15.62:0/2497147443 wait complete. 2024-04-26T22:27:59.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:27:59.393 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:27:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.495+0000 7f6d280ea640 1 Processor -- start 2024-04-26T22:27:59.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 -- start start 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d2015f5e0 0x7f6d2006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d200d2b30 0x7f6d2006e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d200d25f0 con 0x7f6d2006d560 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f6d200cda70 con 0x7f6d2015f5e0 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.496+0000 7f6d280ea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200d2030 con 0x7f6d200d2b30 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25e86640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f6d2015f5e0 0x7f6d2006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d26687640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6d2006d560 0x7f6d200cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36014/0 (socket says 172.21.15.62:36014) 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d26687640 1 -- 172.21.15.62:0/3807094997 learned_addr learned my addr 172.21.15.62:0/3807094997 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25685640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d200d2b30 0x7f6d2006e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25e86640 1 -- 172.21.15.62:0/3807094997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d200d2b30 msgr2=0x7f6d2006e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25e86640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d200d2b30 0x7f6d2006e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25e86640 1 -- 172.21.15.62:0/3807094997 >> v1:172.21.15.62:6789/0 conn(0x7f6d2006d560 legacy=0x7f6d200cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d24e84640 1 -- 172.21.15.62:0/3807094997 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615051981 0 0) 0x7f6d200d25f0 con 0x7f6d2006d560 2024-04-26T22:27:59.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.497+0000 7f6d25e86640 1 -- 172.21.15.62:0/3807094997 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d200d2310 con 0x7f6d2015f5e0 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d25e86640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2015f5e0 0x7f6d2006dd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d180097a0 tx=0x7f6d1802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1f5d9923f96801 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d24e84640 1 -- 172.21.15.62:0/3807094997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18037070 con 0x7f6d2015f5e0 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d24e84640 1 -- 172.21.15.62:0/3807094997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1802ae50 con 0x7f6d2015f5e0 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d24e84640 1 -- 172.21.15.62:0/3807094997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18032660 con 0x7f6d2015f5e0 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 -- 172.21.15.62:0/3807094997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2015f5e0 msgr2=0x7f6d2006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2015f5e0 0x7f6d2006dd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6d180097a0 tx=0x7f6d1802a100 comp rx=0 tx=0).stop 2024-04-26T22:27:59.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 -- 172.21.15.62:0/3807094997 shutdown_connections 2024-04-26T22:27:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d200d2b30 0x7f6d2006e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 --2- 172.21.15.62:0/3807094997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2015f5e0 0x7f6d2006dd60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 -- 172.21.15.62:0/3807094997 >> 172.21.15.62:0/3807094997 conn(0x7f6d200d2f80 msgr2=0x7f6d200d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 -- 172.21.15.62:0/3807094997 shutdown_connections 2024-04-26T22:27:59.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.498+0000 7f6d280ea640 1 -- 172.21.15.62:0/3807094997 wait complete. 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 Processor -- start 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 -- start start 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d200d2b30 0x7f6d20159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d2015f5e0 0x7f6d201556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200cef10 con 0x7f6d2006d560 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200d1cb0 con 0x7f6d200d2b30 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d280ea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6d200cf090 con 0x7f6d2015f5e0 2024-04-26T22:27:59.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d25e86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d25e86640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37332/0 (socket says 172.21.15.62:37332) 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.500+0000 7f6d25e86640 1 -- 172.21.15.62:0/1124477230 learned_addr learned my addr 172.21.15.62:0/1124477230 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d26687640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d2015f5e0 0x7f6d201556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25685640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d200d2b30 0x7f6d20159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 -- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d2015f5e0 msgr2=0x7f6d201556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d2015f5e0 0x7f6d201556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 -- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d200d2b30 msgr2=0x7f6d20159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d200d2b30 0x7f6d20159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 -- 172.21.15.62:0/1124477230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2015a460 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d25e86640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000c00 tx=0x7f6d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18037040 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d18004030 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2015a740 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18004400 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.501+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2015b6a0 con 0x7f6d2006d560 2024-04-26T22:27:59.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.502+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d200693e0 con 0x7f6d2006d560 2024-04-26T22:27:59.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.503+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d18003610 con 0x7f6d2006d560 2024-04-26T22:27:59.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.503+0000 7f6d167fc640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 0x7f6d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.503+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7f6d180ac320 con 0x7f6d2006d560 2024-04-26T22:27:59.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.503+0000 7f6d25685640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 0x7f6d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.504+0000 7f6d25685640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 0x7f6d0406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d2015b500 tx=0x7f6d10009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.507+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6d1804c070 con 0x7f6d2006d560 2024-04-26T22:27:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.693+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d200d0720 con 0x7f6d2006d560 2024-04-26T22:27:59.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.694+0000 7f6d167fc640 1 -- 172.21.15.62:0/1124477230 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f6d1807a570 con 0x7f6d2006d560 2024-04-26T22:27:59.695 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.695+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 msgr2=0x7f6d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.695+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 0x7f6d0406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d2015b500 tx=0x7f6d10009190 comp rx=0 tx=0).stop 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.695+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 msgr2=0x7f6d200cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.695+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d18000c00 tx=0x7f6d180046c0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 shutdown_connections 2024-04-26T22:27:59.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6d0406c8f0 0x7f6d0406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6d2015f5e0 0x7f6d201556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d200d2b30 0x7f6d20159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 --2- 172.21.15.62:0/1124477230 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6d2006d560 0x7f6d200cc870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 >> 172.21.15.62:0/1124477230 conn(0x7f6d200d2f80 msgr2=0x7f6d200cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 shutdown_connections 2024-04-26T22:27:59.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.696+0000 7f6d280ea640 1 -- 172.21.15.62:0/1124477230 wait complete. 2024-04-26T22:27:59.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T22:27:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.806+0000 7fc5a08e1640 1 Processor -- start 2024-04-26T22:27:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.806+0000 7fc5a08e1640 1 -- start start 2024-04-26T22:27:59.808 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.807+0000 7fc5a08e1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc59c0d3c80 0x7fc59c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.807+0000 7fc5a08e1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c0d0210 con 0x7fc59c0d3820 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.807+0000 7fc5a08e1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c0d04f0 con 0x7fc59c162b40 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.807+0000 7fc5a08e1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc59c0d07d0 con 0x7fc59c0d3c80 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc59ad9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc59c0d3c80 0x7fc59c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc599d9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc59c0d3820 0x7fc59c0b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36016/0 (socket says 172.21.15.62:36016) 2024-04-26T22:27:59.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc599d9b640 1 -- 172.21.15.62:0/3359772904 learned_addr learned my addr 172.21.15.62:0/3359772904 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:27:59.810 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc59ad9d640 1 -- 172.21.15.62:0/3359772904 >> v1:172.21.15.174:6789/0 conn(0x7fc59c162b40 legacy=0x7fc59c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc59ad9d640 1 -- 172.21.15.62:0/3359772904 >> v1:172.21.15.62:6789/0 conn(0x7fc59c0d3820 legacy=0x7fc59c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc59ad9d640 1 -- 172.21.15.62:0/3359772904 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c0d0ab0 con 0x7fc59c0d3c80 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.808+0000 7fc59959a640 1 -- 172.21.15.62:0/3359772904 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546189501 0 0) 0x7fc59c0d0210 con 0x7fc59c0d3820 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc59ad9d640 1 --2- 172.21.15.62:0/3359772904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0b52b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc5940097a0 tx=0x7fc59402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0985c1f833181f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc59959a640 1 -- 172.21.15.62:0/3359772904 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594037070 con 0x7fc59c0d3c80 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc59959a640 1 -- 172.21.15.62:0/3359772904 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc594032440 con 0x7fc59c0d3c80 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc59959a640 1 -- 172.21.15.62:0/3359772904 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594032810 con 0x7fc59c0d3c80 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 -- 172.21.15.62:0/3359772904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3c80 msgr2=0x7fc59c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/3359772904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0b52b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc5940097a0 tx=0x7fc59402a1e0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 -- 172.21.15.62:0/3359772904 shutdown_connections 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/3359772904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0b52b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 -- 172.21.15.62:0/3359772904 >> 172.21.15.62:0/3359772904 conn(0x7fc59c09e550 msgr2=0x7fc59c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.809+0000 7fc5a08e1640 1 -- 172.21.15.62:0/3359772904 shutdown_connections 2024-04-26T22:27:59.811 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.810+0000 7fc5a08e1640 1 -- 172.21.15.62:0/3359772904 wait complete. 2024-04-26T22:27:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.811+0000 7fc5a08e1640 1 Processor -- start 2024-04-26T22:27:59.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.811+0000 7fc5a08e1640 1 -- start start 2024-04-26T22:27:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 0x7fc59c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0d12b0 con 0x7fc59c0d3820 2024-04-26T22:27:59.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c0d4110 con 0x7fc59c0d3c80 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c0cfe30 con 0x7fc59c162b40 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37362/0 (socket says 172.21.15.62:37362) 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 -- 172.21.15.62:0/1254602608 learned_addr learned my addr 172.21.15.62:0/1254602608 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc599d9b640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59ad9d640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 0x7fc59c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 -- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 msgr2=0x7fc59c0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 0x7fc59c0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 -- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 msgr2=0x7fc59c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 -- 172.21.15.62:0/1254602608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1f9280 con 0x7fc59c0d3820 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59ad9d640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 0x7fc59c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc599d9b640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:27:59.815 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc59a59c640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c7c0 tx=0x7fc58c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c016020 con 0x7fc59c0d3820 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c00a9f0 con 0x7fc59c0d3820 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c0ca970 con 0x7fc59c0d3820 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00ae20 con 0x7fc59c0d3820 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.812+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c0c9080 con 0x7fc59c0d3820 2024-04-26T22:27:59.816 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.813+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c0ca790 con 0x7fc59c0d3820 2024-04-26T22:27:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.814+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc58c019020 con 0x7fc59c0d3820 2024-04-26T22:27:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.814+0000 7fc592ffd640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 0x7fc57c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:27:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.814+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fc58c08b440 con 0x7fc59c0d3820 2024-04-26T22:27:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.815+0000 7fc599d9b640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 0x7fc57c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:27:59.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.815+0000 7fc599d9b640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 0x7fc57c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0b8210 tx=0x7fc584009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:27:59.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:27:59.819+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc59c0ca790 con 0x7fc59c0d3820 2024-04-26T22:28:00.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.010+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc59c06a360 con 0x7fc59c0d3820 2024-04-26T22:28:00.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.010+0000 7fc592ffd640 1 -- 172.21.15.62:0/1254602608 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) ==== 84+0+65 (secure 0 0 0) 0x7fc58c020070 con 0x7fc59c0d3820 2024-04-26T22:28:00.011 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:00.012 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 msgr2=0x7fc57c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 0x7fc57c06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0b8210 tx=0x7fc584009040 comp rx=0 tx=0).stop 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 msgr2=0x7fc59c0c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c7c0 tx=0x7fc58c00cc80 comp rx=0 tx=0).stop 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 shutdown_connections 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc57c06c920 0x7fc57c06ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.012+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc59c162b40 0x7fc59c0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.013+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc59c0d3c80 0x7fc59c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.013+0000 7fc5a08e1640 1 --2- 172.21.15.62:0/1254602608 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc59c0d3820 0x7fc59c0c8b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.013+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 >> 172.21.15.62:0/1254602608 conn(0x7fc59c09e550 msgr2=0x7fc59c0b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:00.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.013+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 shutdown_connections 2024-04-26T22:28:00.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.013+0000 7fc5a08e1640 1 -- 172.21.15.62:0/1254602608 wait complete. 2024-04-26T22:28:00.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T22:28:00.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T22:28:00.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.128+0000 7fa84cd14640 1 Processor -- start 2024-04-26T22:28:00.129 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.128+0000 7fa84cd14640 1 -- start start 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa84cd14640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 0x7fa8480ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa84cd14640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa84806d590 0x7fa8480b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa84cd14640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa848070720 con 0x7fa84815f350 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa84cd14640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa84815f730 con 0x7fa84806d590 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa84cd14640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa848163de0 con 0x7fa8480683f0 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa846d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa84806d590 0x7fa8480b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa846d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa84806d590 0x7fa8480b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37370/0 (socket says 172.21.15.62:37370) 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa846d9d640 1 -- 172.21.15.62:0/3376318647 learned_addr learned my addr 172.21.15.62:0/3376318647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.129+0000 7fa845d9b640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 0x7fa8480ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa846d9d640 1 -- 172.21.15.62:0/3376318647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 msgr2=0x7fa8480ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa846d9d640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 0x7fa8480ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa846d9d640 1 -- 172.21.15.62:0/3376318647 >> v1:172.21.15.174:6789/0 conn(0x7fa84815f350 legacy=0x7fa84806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa846d9d640 1 -- 172.21.15.62:0/3376318647 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84806c3e0 con 0x7fa84806d590 2024-04-26T22:28:00.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa845d9b640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 0x7fa8480ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:00.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa846d9d640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84806d590 0x7fa8480b44f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa84806c5b0 tx=0x7fa83802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bdec13d0b477777 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa84559a640 1 -- 172.21.15.62:0/3376318647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838036070 con 0x7fa84806d590 2024-04-26T22:28:00.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa84559a640 1 -- 172.21.15.62:0/3376318647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838031600 con 0x7fa84806d590 2024-04-26T22:28:00.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.130+0000 7fa84559a640 1 -- 172.21.15.62:0/3376318647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8380319d0 con 0x7fa84806d590 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 -- 172.21.15.62:0/3376318647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84806d590 msgr2=0x7fa8480b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84806d590 0x7fa8480b44f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa84806c5b0 tx=0x7fa83802a710 comp rx=0 tx=0).stop 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 -- 172.21.15.62:0/3376318647 shutdown_connections 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa84806d590 0x7fa8480b44f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 --2- 172.21.15.62:0/3376318647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8480683f0 0x7fa8480ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 -- 172.21.15.62:0/3376318647 >> 172.21.15.62:0/3376318647 conn(0x7fa8480d15c0 msgr2=0x7fa8480d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 -- 172.21.15.62:0/3376318647 shutdown_connections 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.131+0000 7fa84cd14640 1 -- 172.21.15.62:0/3376318647 wait complete. 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.132+0000 7fa84cd14640 1 Processor -- start 2024-04-26T22:28:00.134 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.132+0000 7fa84cd14640 1 -- start start 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 0x7fa848150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 0x7fa848160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa848163b00 con 0x7fa8480683f0 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa84806c9f0 con 0x7fa84806d590 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84cd14640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa84806c680 con 0x7fa84815f350 2024-04-26T22:28:00.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37386/0 (socket says 172.21.15.62:37386) 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa84659c640 1 -- 172.21.15.62:0/4270871551 learned_addr learned my addr 172.21.15.62:0/4270871551 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa845d9b640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 0x7fa848150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.133+0000 7fa846d9d640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 0x7fa848160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 -- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 msgr2=0x7fa848160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 0x7fa848160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 -- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 msgr2=0x7fa848150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 0x7fa848150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 -- 172.21.15.62:0/4270871551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa848151990 con 0x7fa8480683f0 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa845d9b640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 0x7fa848150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa846d9d640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 0x7fa848160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:00.137 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84659c640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c3c0 tx=0x7fa84000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400052d0 con 0x7fa8480683f0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840005490 con 0x7fa8480683f0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa848151c70 con 0x7fa8480683f0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840007260 con 0x7fa8480683f0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.134+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa848152230 con 0x7fa8480683f0 2024-04-26T22:28:00.138 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.135+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa848069de0 con 0x7fa8480683f0 2024-04-26T22:28:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.136+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa840007400 con 0x7fa8480683f0 2024-04-26T22:28:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.136+0000 7fa83effd640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.136+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4857+0+0 (secure 0 0 0) 0x7fa84008acb0 con 0x7fa8480683f0 2024-04-26T22:28:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.136+0000 7fa845d9b640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.139 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.137+0000 7fa845d9b640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 0x7fa82806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa84815ce00 tx=0x7fa830008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.140+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa848069de0 con 0x7fa8480683f0 2024-04-26T22:28:00.325 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.324+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa84806a360 con 0x7fa8480683f0 2024-04-26T22:28:00.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.527+0000 7fa83effd640 1 -- 172.21.15.62:0/4270871551 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) ==== 138+0+0 (secure 0 0 0) 0x7fa848070720 con 0x7fa8480683f0 2024-04-26T22:28:00.528 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.529+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 msgr2=0x7fa82806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.529+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 0x7fa82806ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa84815ce00 tx=0x7fa830008040 comp rx=0 tx=0).stop 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.529+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 msgr2=0x7fa848156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.529+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa84000c3c0 tx=0x7fa84000c880 comp rx=0 tx=0).stop 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 shutdown_connections 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa82806c920 0x7fa82806ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa84815f350 0x7fa848160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa84806d590 0x7fa848150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 --2- 172.21.15.62:0/4270871551 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8480683f0 0x7fa848156ec0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 >> 172.21.15.62:0/4270871551 conn(0x7fa8480d15c0 msgr2=0x7fa848162af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 shutdown_connections 2024-04-26T22:28:00.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.530+0000 7fa84cd14640 1 -- 172.21.15.62:0/4270871551 wait complete. 2024-04-26T22:28:00.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T22:28:00.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.653+0000 7fdf12d6a640 1 Processor -- start 2024-04-26T22:28:00.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.653+0000 7fdf12d6a640 1 -- start start 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf12d6a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf12d6a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c073a20 0x7fdf0c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf12d6a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf0c06f750 con 0x7fdf0c0cb6f0 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf12d6a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fdf0c0cbef0 con 0x7fdf0c15d830 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf12d6a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c073e50 con 0x7fdf0c073a20 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37388/0 (socket says 172.21.15.62:37388) 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 -- 172.21.15.62:0/1877926683 learned_addr learned my addr 172.21.15.62:0/1877926683 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:00.657 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf11307640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c073a20 0x7fdf0c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 -- 172.21.15.62:0/1877926683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c073a20 msgr2=0x7fdf0c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c073a20 0x7fdf0c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 -- 172.21.15.62:0/1877926683 >> v1:172.21.15.174:6789/0 conn(0x7fdf0c0cb6f0 legacy=0x7fdf0c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.654+0000 7fdf10b06640 1 -- 172.21.15.62:0/1877926683 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c06fa30 con 0x7fdf0c15d830 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.655+0000 7fdf10b06640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf000098a0 tx=0x7fdf0002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8451dfe2a53df23e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.655+0000 7fdf0b7fe640 1 -- 172.21.15.62:0/1877926683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00037070 con 0x7fdf0c15d830 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.655+0000 7fdf0b7fe640 1 -- 172.21.15.62:0/1877926683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf00032440 con 0x7fdf0c15d830 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.655+0000 7fdf0b7fe640 1 -- 172.21.15.62:0/1877926683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf00032810 con 0x7fdf0c15d830 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 -- 172.21.15.62:0/1877926683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 msgr2=0x7fdf0c0cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf000098a0 tx=0x7fdf0002a290 comp rx=0 tx=0).stop 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 -- 172.21.15.62:0/1877926683 shutdown_connections 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c073a20 0x7fdf0c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.658 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/1877926683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c0cc0f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 -- 172.21.15.62:0/1877926683 >> 172.21.15.62:0/1877926683 conn(0x7fdf0c0d0d80 msgr2=0x7fdf0c0d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 -- 172.21.15.62:0/1877926683 shutdown_connections 2024-04-26T22:28:00.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.656+0000 7fdf12d6a640 1 -- 172.21.15.62:0/1877926683 wait complete. 2024-04-26T22:28:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.657+0000 7fdf12d6a640 1 Processor -- start 2024-04-26T22:28:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.657+0000 7fdf12d6a640 1 -- start start 2024-04-26T22:28:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf0c073a20 0x7fdf0c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 0x7fdf0c0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c06e300 con 0x7fdf0c15d830 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c15e820 con 0x7fdf0c073a20 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf12d6a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0c06dfb0 con 0x7fdf0c0cb6f0 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37396/0 (socket says 172.21.15.62:37396) 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 -- 172.21.15.62:0/166704385 learned_addr learned my addr 172.21.15.62:0/166704385 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf10b06640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf0c073a20 0x7fdf0c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf0bfff640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 0x7fdf0c0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 -- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 msgr2=0x7fdf0c0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 0x7fdf0c0c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 -- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf0c073a20 msgr2=0x7fdf0c0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf0c073a20 0x7fdf0c0ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf11307640 1 -- 172.21.15.62:0/166704385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c0c6c80 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.658+0000 7fdf0bfff640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 0x7fdf0c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf11307640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf04007e40 tx=0x7fdf0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04014620 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf04019070 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04014a20 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c0c69a0 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.659+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c0c9710 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.660+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c0c9250 con 0x7fdf0c15d830 2024-04-26T22:28:00.663 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.660+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf04014bc0 con 0x7fdf0c15d830 2024-04-26T22:28:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.661+0000 7fdf097fa640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 0x7fdef006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.661+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5227+0+0 (secure 0 0 0) 0x7fdf0408cfe0 con 0x7fdf0c15d830 2024-04-26T22:28:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.661+0000 7fdf10b06640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 0x7fdef006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:00.664 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.662+0000 7fdf10b06640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 0x7fdef006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf00034000 tx=0x7fdf0002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:00.666 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.665+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdf0c0c9250 con 0x7fdf0c15d830 2024-04-26T22:28:00.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:00.853+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdf0c0d17c0 con 0x7fdf0c15d830 2024-04-26T22:28:01.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.534+0000 7fdf097fa640 1 -- 172.21.15.62:0/166704385 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) ==== 170+0+0 (secure 0 0 0) 0x7fdf0c0d17c0 con 0x7fdf0c15d830 2024-04-26T22:28:01.536 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-26T22:28:01.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 msgr2=0x7fdef006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 0x7fdef006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf00034000 tx=0x7fdf0002e000 comp rx=0 tx=0).stop 2024-04-26T22:28:01.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 msgr2=0x7fdf0c1f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdf04007e40 tx=0x7fdf0400e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 shutdown_connections 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fdef006c8f0 0x7fdef006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdf0c15d830 0x7fdf0c1f91e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdf0c0cb6f0 0x7fdf0c0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 --2- 172.21.15.62:0/166704385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf0c073a20 0x7fdf0c0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 >> 172.21.15.62:0/166704385 conn(0x7fdf0c0d0d80 msgr2=0x7fdf0c0afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 shutdown_connections 2024-04-26T22:28:01.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.537+0000 7fdf12d6a640 1 -- 172.21.15.62:0/166704385 wait complete. 2024-04-26T22:28:01.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T22:28:01.668 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.667+0000 7fca0659e640 1 Processor -- start 2024-04-26T22:28:01.669 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca0659e640 1 -- start start 2024-04-26T22:28:01.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca0659e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fca000b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca0659e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca000cd640 con 0x7fca000d26f0 2024-04-26T22:28:01.671 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca0659e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca000cd920 con 0x7fca000cf870 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca0659e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fca000cdc00 con 0x7fca000b2290 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca04b3b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fca000d26f0 0x7fca0015ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36030/0 (socket says 172.21.15.62:36030) 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.668+0000 7fca04b3b640 1 -- 172.21.15.62:0/1002051738 learned_addr learned my addr 172.21.15.62:0/1002051738 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9fffff640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fca000b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2299054570 0 0) 0x7fca000cd640 con 0x7fca000d26f0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9e8003520 con 0x7fca000d26f0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4168546617 0 0) 0x7fc9e8003520 con 0x7fca000d26f0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 msgr2=0x7fca000b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fca000b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 >> v1:172.21.15.174:6789/0 conn(0x7fca000cf870 legacy=0x7fca000d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca000cdee0 con 0x7fca000d26f0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fc9f80019f0 con 0x7fca000d26f0 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 >> v1:172.21.15.62:6789/0 conn(0x7fca000d26f0 legacy=0x7fca0015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.672 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e80039e0 0x7fc9e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9e8004390 0x7fc9e8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fc9e80050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fca000cdc00 con 0x7fc9e8004390 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e800b9c0 con 0x7fc9e80039e0 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e800bbc0 con 0x7fca000b2290 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9e8004390 0x7fc9e8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9fffff640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e80039e0 0x7fc9e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fca04b3b640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fc9e80050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 msgr2=0x7fc9e80050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000b2290 0x7fc9e80050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e80039e0 msgr2=0x7fc9e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e80039e0 0x7fc9e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.669+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/1002051738 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e8003520 con 0x7fc9e8004390 2024-04-26T22:28:01.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.670+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9e8004390 0x7fc9e8004860 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4001860 tx=0x7fc9e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9fb6dc16bec3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.670+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4037070 con 0x7fc9e8004390 2024-04-26T22:28:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.670+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f402abf0 con 0x7fc9e8004390 2024-04-26T22:28:01.674 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.670+0000 7fc9feffd640 1 -- 172.21.15.62:0/1002051738 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4001430 con 0x7fc9e8004390 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 -- 172.21.15.62:0/1002051738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9e8004390 msgr2=0x7fc9e8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 --2- 172.21.15.62:0/1002051738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc9e8004390 0x7fc9e8004860 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4001860 tx=0x7fc9e8005830 comp rx=0 tx=0).stop 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca04b3b640 1 -- 172.21.15.62:0/1002051738 reap_dead start 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 -- 172.21.15.62:0/1002051738 shutdown_connections 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 -- 172.21.15.62:0/1002051738 >> 172.21.15.62:0/1002051738 conn(0x7fca000d2ff0 msgr2=0x7fca000d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 -- 172.21.15.62:0/1002051738 shutdown_connections 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.671+0000 7fca0659e640 1 -- 172.21.15.62:0/1002051738 wait complete. 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.672+0000 7fca0659e640 1 Processor -- start 2024-04-26T22:28:01.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.672+0000 7fca0659e640 1 -- start start 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e8004390 0x7fca0015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 0x7fca000c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fca000b2930 con 0x7fca000c8920 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca000b2670 con 0x7fc9e8004390 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca0659e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fca000b2ab0 con 0x7fca000c6d10 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37416/0 (socket says 172.21.15.62:37416) 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/3295284902 learned_addr learned my addr 172.21.15.62:0/3295284902 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9fffff640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e8004390 0x7fca0015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca04b3b640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 0x7fca000c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 msgr2=0x7fca000c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 0x7fca000c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.678 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e8004390 msgr2=0x7fca0015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e8004390 0x7fca0015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fc9ff7fe640 1 -- 172.21.15.62:0/3295284902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0015c8a0 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.673+0000 7fca04b3b640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 0x7fca000c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fc9ff7fe640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4001950 tx=0x7fc9f4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f403f020 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f4005270 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca000c7ab0 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4037040 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.674+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca000c8070 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.675+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca000d3550 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.675+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9f4042020 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.676+0000 7fc9fcff9640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 0x7fc9ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.676+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5609+0+0 (secure 0 0 0) 0x7fc9f40ab7c0 con 0x7fca000c8920 2024-04-26T22:28:01.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.676+0000 7fc9fffff640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 0x7fc9ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:01.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.676+0000 7fc9fffff640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 0x7fc9ec06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009b00 tx=0x7fc9f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:01.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.680+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca000d3550 con 0x7fca000c8920 2024-04-26T22:28:01.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:01.908+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fca0006a360 con 0x7fca000c8920 2024-04-26T22:28:02.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.546+0000 7fc9fcff9640 1 -- 172.21.15.62:0/3295284902 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fc9f4049030 con 0x7fca000c8920 2024-04-26T22:28:02.548 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T22:28:02.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.558+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 msgr2=0x7fc9ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.558+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 0x7fc9ec06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009b00 tx=0x7fc9f0008040 comp rx=0 tx=0).stop 2024-04-26T22:28:02.559 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.558+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 msgr2=0x7fca000c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.560 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.558+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4001950 tx=0x7fc9f4001230 comp rx=0 tx=0).stop 2024-04-26T22:28:02.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.562+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 shutdown_connections 2024-04-26T22:28:02.563 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.562+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc9ec06c920 0x7fc9ec06ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.562+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca000c6d10 0x7fca000c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.562+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca000c8920 0x7fca000c8da0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.562+0000 7fca0659e640 1 --2- 172.21.15.62:0/3295284902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9e8004390 0x7fca0015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.563+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 >> 172.21.15.62:0/3295284902 conn(0x7fca000d2ff0 msgr2=0x7fca0006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.563+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 shutdown_connections 2024-04-26T22:28:02.564 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.564+0000 7fca0659e640 1 -- 172.21.15.62:0/3295284902 wait complete. 2024-04-26T22:28:02.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:02.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 Processor -- start 2024-04-26T22:28:02.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 -- start start 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40b015f4f0 0x7f40b006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b00b1480 0x7f40b00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b00b59b0 con 0x7f40b006c240 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f40b0070350 con 0x7f40b015f4f0 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.689+0000 7f40b4eb0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f40b00b5c70 con 0x7f40b00b1480 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40b015f4f0 0x7f40b006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f40b015f4f0 0x7f40b006fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37428/0 (socket says 172.21.15.62:37428) 2024-04-26T22:28:02.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40ae59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f40b006c240 0x7f40b006c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:36034/0 (socket says 172.21.15.62:36034) 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 -- 172.21.15.62:0/4290595151 learned_addr learned my addr 172.21.15.62:0/4290595151 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40aed9d640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b00b1480 0x7f40b00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 -- 172.21.15.62:0/4290595151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b00b1480 msgr2=0x7f40b00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b00b1480 0x7f40b00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 -- 172.21.15.62:0/4290595151 >> v1:172.21.15.62:6789/0 conn(0x7f40b006c240 legacy=0x7f40b006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40add9b640 1 -- 172.21.15.62:0/4290595151 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b00b5f50 con 0x7f40b015f4f0 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.690+0000 7f40ad59a640 1 -- 172.21.15.62:0/4290595151 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477570730 0 0) 0x7f40b00b59b0 con 0x7f40b006c240 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40add9b640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b006fd70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62120c6795bc47a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40ad59a640 1 -- 172.21.15.62:0/4290595151 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0037070 con 0x7f40b015f4f0 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40ad59a640 1 -- 172.21.15.62:0/4290595151 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a0032440 con 0x7f40b015f4f0 2024-04-26T22:28:02.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40ad59a640 1 -- 172.21.15.62:0/4290595151 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0032810 con 0x7f40b015f4f0 2024-04-26T22:28:02.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 -- 172.21.15.62:0/4290595151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 msgr2=0x7f40b006fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b006fd70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002a270 comp rx=0 tx=0).stop 2024-04-26T22:28:02.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 -- 172.21.15.62:0/4290595151 shutdown_connections 2024-04-26T22:28:02.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b006fd70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/4290595151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b00b1480 0x7f40b00ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 -- 172.21.15.62:0/4290595151 >> 172.21.15.62:0/4290595151 conn(0x7f40b00b3b70 msgr2=0x7f40b00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.691+0000 7f40b4eb0640 1 -- 172.21.15.62:0/4290595151 shutdown_connections 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.692+0000 7f40b4eb0640 1 -- 172.21.15.62:0/4290595151 wait complete. 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.693+0000 7f40b4eb0640 1 Processor -- start 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.693+0000 7f40b4eb0640 1 -- start start 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b006c240 0x7f40b0145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b00b1480 0x7f40b0158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f40b00b0fc0 con 0x7f40b015f4f0 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40b00aee00 con 0x7f40b00b1480 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f40b00b6750 con 0x7f40b006c240 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:37438/0 (socket says 172.21.15.62:37438) 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 -- 172.21.15.62:0/3129701773 learned_addr learned my addr 172.21.15.62:0/3129701773 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40add9b640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b00b1480 0x7f40b0158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40ae59c640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b006c240 0x7f40b0145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 -- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b006c240 msgr2=0x7f40b0145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b006c240 0x7f40b0145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 -- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b00b1480 msgr2=0x7f40b0158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b00b1480 0x7f40b0158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 -- 172.21.15.62:0/3129701773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b00cefd0 con 0x7f40b015f4f0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40aed9d640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007e30 tx=0x7f40a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014620 con 0x7f40b015f4f0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8019070 con 0x7f40b015f4f0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014a00 con 0x7f40b015f4f0 2024-04-26T22:28:02.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b00cf2b0 con 0x7f40b015f4f0 2024-04-26T22:28:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.694+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b00cf870 con 0x7f40b015f4f0 2024-04-26T22:28:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.695+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b006a360 con 0x7f40b015f4f0 2024-04-26T22:28:02.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.696+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40a8014ba0 con 0x7f40b015f4f0 2024-04-26T22:28:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.696+0000 7f40a6ffd640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 0x7f409006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.696+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f40a808c440 con 0x7f40b015f4f0 2024-04-26T22:28:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.697+0000 7f40ae59c640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 0x7f409006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:02.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.697+0000 7f40ae59c640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 0x7f409006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4098000f60 tx=0x7f4098007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:02.702 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.702+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40a805a870 con 0x7f40b015f4f0 2024-04-26T22:28:02.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.888+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f40b00b6fd0 con 0x7f40b015f4f0 2024-04-26T22:28:02.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.889+0000 7f40a6ffd640 1 -- 172.21.15.62:0/3129701773 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7f40a805a170 con 0x7f40b015f4f0 2024-04-26T22:28:02.891 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:02.891 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:01.910932+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 msgr2=0x7f409006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 0x7f409006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4098000f60 tx=0x7f4098007b50 comp rx=0 tx=0).stop 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 msgr2=0x7f40b0158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007e30 tx=0x7f40a800e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 shutdown_connections 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f409006c8f0 0x7f409006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f40b015f4f0 0x7f40b0158760 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b00b1480 0x7f40b0158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 --2- 172.21.15.62:0/3129701773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f40b006c240 0x7f40b0145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:02.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 >> 172.21.15.62:0/3129701773 conn(0x7f40b00b3b70 msgr2=0x7f40b015daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:02.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 shutdown_connections 2024-04-26T22:28:02.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:02.892+0000 7f40b4eb0640 1 -- 172.21.15.62:0/3129701773 wait complete. 2024-04-26T22:28:02.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:03.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.003+0000 7ff585ad1640 1 Processor -- start 2024-04-26T22:28:03.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.003+0000 7ff585ad1640 1 -- start start 2024-04-26T22:28:03.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff585ad1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff58006ea40 0x7ff5800d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff585ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff585ad1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5800b1d70 con 0x7ff580162e40 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff585ad1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff580163220 con 0x7ff58006ea40 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff585ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff5800d01c0 con 0x7ff5800cf830 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:44174/0 (socket says 172.21.15.62:44174) 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 -- 172.21.15.62:0/1487400278 learned_addr learned my addr 172.21.15.62:0/1487400278 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57effd640 1 --2- 172.21.15.62:0/1487400278 >> v2:172.21.15.174:3300/0 conn(0x7ff58006ea40 0x7ff5800d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 -- 172.21.15.62:0/1487400278 >> v2:172.21.15.174:3300/0 conn(0x7ff58006ea40 msgr2=0x7ff5800d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 --2- 172.21.15.62:0/1487400278 >> v2:172.21.15.174:3300/0 conn(0x7ff58006ea40 0x7ff5800d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 -- 172.21.15.62:0/1487400278 >> v1:172.21.15.174:6789/0 conn(0x7ff580162e40 legacy=0x7ff5800b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57e7fc640 1 -- 172.21.15.62:0/1487400278 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532032924 0 0) 0x7ff5800b1d70 con 0x7ff580162e40 2024-04-26T22:28:03.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.004+0000 7ff57ffff640 1 -- 172.21.15.62:0/1487400278 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5800d1960 con 0x7ff5800cf830 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.005+0000 7ff57ffff640 1 --2- 172.21.15.62:0/1487400278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff578009e50 tx=0x7ff57802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea43b96f5f7b6b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.005+0000 7ff57e7fc640 1 -- 172.21.15.62:0/1487400278 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578036070 con 0x7ff5800cf830 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.005+0000 7ff57e7fc640 1 -- 172.21.15.62:0/1487400278 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578031600 con 0x7ff5800cf830 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.005+0000 7ff57e7fc640 1 -- 172.21.15.62:0/1487400278 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780319d0 con 0x7ff5800cf830 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 -- 172.21.15.62:0/1487400278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 msgr2=0x7ff5800cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 --2- 172.21.15.62:0/1487400278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff578009e50 tx=0x7ff57802a680 comp rx=0 tx=0).stop 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 -- 172.21.15.62:0/1487400278 shutdown_connections 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 --2- 172.21.15.62:0/1487400278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5800cf830 0x7ff5800cfc80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 --2- 172.21.15.62:0/1487400278 >> v2:172.21.15.174:3300/0 conn(0x7ff58006ea40 0x7ff5800d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 -- 172.21.15.62:0/1487400278 >> 172.21.15.62:0/1487400278 conn(0x7ff58009ba50 msgr2=0x7ff58009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 -- 172.21.15.62:0/1487400278 shutdown_connections 2024-04-26T22:28:03.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.006+0000 7ff585ad1640 1 -- 172.21.15.62:0/1487400278 wait complete. 2024-04-26T22:28:03.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.007+0000 7ff585ad1640 1 Processor -- start 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.007+0000 7ff585ad1640 1 -- start start 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5800cf830 0x7ff5800cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff580162e40 0x7ff5800b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800d1f20 con 0x7ff58006ea40 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800d0eb0 con 0x7ff5800cf830 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff585ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff5800ce010 con 0x7ff580162e40 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff57f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff57f7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43286/0 (socket says 172.21.15.62:43286) 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff57f7fe640 1 -- 172.21.15.62:0/2151638477 learned_addr learned my addr 172.21.15.62:0/2151638477 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff57ffff640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff580162e40 0x7ff5800b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.008+0000 7ff57effd640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5800cf830 0x7ff5800cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 -- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff580162e40 msgr2=0x7ff5800b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff580162e40 0x7ff5800b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 -- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5800cf830 msgr2=0x7ff5800cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5800cf830 0x7ff5800cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 -- 172.21.15.62:0/2151638477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5800ca460 con 0x7ff58006ea40 2024-04-26T22:28:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff57f7fe640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c410 tx=0x7ff57000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570017070 con 0x7ff58006ea40 2024-04-26T22:28:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5700035f0 con 0x7ff58006ea40 2024-04-26T22:28:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5800ca740 con 0x7ff58006ea40 2024-04-26T22:28:03.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5700039e0 con 0x7ff58006ea40 2024-04-26T22:28:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.009+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5800cad00 con 0x7ff58006ea40 2024-04-26T22:28:03.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.010+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58006a360 con 0x7ff58006ea40 2024-04-26T22:28:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.010+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff57001a070 con 0x7ff58006ea40 2024-04-26T22:28:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.011+0000 7ff567fff640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.011+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7ff57008af50 con 0x7ff58006ea40 2024-04-26T22:28:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.011+0000 7ff57effd640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.015 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.012+0000 7ff57effd640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5800d08d0 tx=0x7ff574009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.016+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff570058cb0 con 0x7ff58006ea40 2024-04-26T22:28:03.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.202+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5800ccde0 con 0x7ff58006ea40 2024-04-26T22:28:03.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.203+0000 7ff567fff640 1 -- 172.21.15.62:0/2151638477 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16682 (secure 0 0 0) 0x7ff570076e20 con 0x7ff58006ea40 2024-04-26T22:28:03.203 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:03.204 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":30,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:01.910932+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 msgr2=0x7ff56c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 0x7ff56c06edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff5800d08d0 tx=0x7ff574009190 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 msgr2=0x7ff5801f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c410 tx=0x7ff57000c8d0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 shutdown_connections 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff56c06c8f0 0x7ff56c06edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff580162e40 0x7ff5800b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5800cf830 0x7ff5800cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 --2- 172.21.15.62:0/2151638477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff58006ea40 0x7ff5801f4950 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 >> 172.21.15.62:0/2151638477 conn(0x7ff58009ba50 msgr2=0x7ff5800b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 shutdown_connections 2024-04-26T22:28:03.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.205+0000 7ff585ad1640 1 -- 172.21.15.62:0/2151638477 wait complete. 2024-04-26T22:28:03.217 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T22:28:03.320 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.320+0000 7fca52472640 1 Processor -- start 2024-04-26T22:28:03.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.320+0000 7fca52472640 1 -- start start 2024-04-26T22:28:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca52472640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 0x7fca4c0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca52472640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca4c0729f0 0x7fca4c072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca52472640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4c15c830 con 0x7fca4c15c450 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca52472640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fca4c15d600 con 0x7fca4c0729f0 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca52472640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fca4c15d8e0 con 0x7fca4c06df90 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca4c0729f0 0x7fca4c072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fca4c0729f0 0x7fca4c072dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43290/0 (socket says 172.21.15.62:43290) 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 -- 172.21.15.62:0/1598428605 learned_addr learned my addr 172.21.15.62:0/1598428605 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca4bfff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fca4c15c450 0x7fca4c0cbcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42332/0 (socket says 172.21.15.62:42332) 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 -- 172.21.15.62:0/1598428605 >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 msgr2=0x7fca4c0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca4b7fe640 1 --2- 172.21.15.62:0/1598428605 >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 0x7fca4c0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 --2- 172.21.15.62:0/1598428605 >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 0x7fca4c0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca4affd640 1 -- 172.21.15.62:0/1598428605 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621525584 0 0) 0x7fca4c15c830 con 0x7fca4c15c450 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 -- 172.21.15.62:0/1598428605 >> v1:172.21.15.62:6789/0 conn(0x7fca4c15c450 legacy=0x7fca4c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca50a0f640 1 -- 172.21.15.62:0/1598428605 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c06f830 con 0x7fca4c0729f0 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.321+0000 7fca4b7fe640 1 --2- 172.21.15.62:0/1598428605 >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 0x7fca4c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:03.323 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.322+0000 7fca50a0f640 1 --2- 172.21.15.62:0/1598428605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c0729f0 0x7fca4c072dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fca4c050d60 tx=0x7fca4402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56d47487d140f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.322+0000 7fca4affd640 1 -- 172.21.15.62:0/1598428605 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44036070 con 0x7fca4c0729f0 2024-04-26T22:28:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.322+0000 7fca4affd640 1 -- 172.21.15.62:0/1598428605 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca44031600 con 0x7fca4c0729f0 2024-04-26T22:28:03.324 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.322+0000 7fca4affd640 1 -- 172.21.15.62:0/1598428605 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440319d0 con 0x7fca4c0729f0 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 -- 172.21.15.62:0/1598428605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c0729f0 msgr2=0x7fca4c072dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 --2- 172.21.15.62:0/1598428605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c0729f0 0x7fca4c072dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fca4c050d60 tx=0x7fca4402a6a0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 -- 172.21.15.62:0/1598428605 shutdown_connections 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 --2- 172.21.15.62:0/1598428605 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c0729f0 0x7fca4c072dd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 --2- 172.21.15.62:0/1598428605 >> v2:172.21.15.174:3300/0 conn(0x7fca4c06df90 0x7fca4c0cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 -- 172.21.15.62:0/1598428605 >> 172.21.15.62:0/1598428605 conn(0x7fca4c0d0d70 msgr2=0x7fca4c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 -- 172.21.15.62:0/1598428605 shutdown_connections 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.323+0000 7fca52472640 1 -- 172.21.15.62:0/1598428605 wait complete. 2024-04-26T22:28:03.327 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.324+0000 7fca52472640 1 Processor -- start 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.324+0000 7fca52472640 1 -- start start 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca4c06df90 0x7fca4c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 0x7fca4c0cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c0ccaa0 con 0x7fca4c15c450 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c0cc920 con 0x7fca4c0729f0 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca52472640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c073310 con 0x7fca4c06df90 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43294/0 (socket says 172.21.15.62:43294) 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 -- 172.21.15.62:0/2632810331 learned_addr learned my addr 172.21.15.62:0/2632810331 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca4b7fe640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 0x7fca4c0cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca4bfff640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca4c06df90 0x7fca4c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 -- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca4c06df90 msgr2=0x7fca4c155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca4c06df90 0x7fca4c155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.328 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 -- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 msgr2=0x7fca4c0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 0x7fca4c0cfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 -- 172.21.15.62:0/2632810331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0d0870 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca4b7fe640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 0x7fca4c0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.325+0000 7fca50a0f640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca44034040 tx=0x7fca44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.326+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44036050 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.326+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca44042070 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.326+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c0c8150 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.326+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44009510 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.326+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1f3470 con 0x7fca4c15c450 2024-04-26T22:28:03.329 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.327+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c06a360 con 0x7fca4c15c450 2024-04-26T22:28:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.327+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca44007bd0 con 0x7fca4c15c450 2024-04-26T22:28:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.328+0000 7fca48ff9640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 0x7fca3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.328+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fca440acb20 con 0x7fca4c15c450 2024-04-26T22:28:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.328+0000 7fca4bfff640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 0x7fca3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:03.330 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.328+0000 7fca4bfff640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 0x7fca3806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca40000f60 tx=0x7fca40007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:03.332 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.332+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca4407a900 con 0x7fca4c15c450 2024-04-26T22:28:03.561 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:03.561+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fca4c0b6cf0 con 0x7fca4c15c450 2024-04-26T22:28:04.567 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.567+0000 7fca48ff9640 1 -- 172.21.15.62:0/2632810331 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fca4407ec00 con 0x7fca4c15c450 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 msgr2=0x7fca3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 0x7fca3806edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fca40000f60 tx=0x7fca40007c00 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 msgr2=0x7fca4c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fca44034040 tx=0x7fca44004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 shutdown_connections 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fca3806c8f0 0x7fca3806edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fca4c15c450 0x7fca4c0d0240 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c0729f0 0x7fca4c0cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 --2- 172.21.15.62:0/2632810331 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fca4c06df90 0x7fca4c155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.570+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 >> 172.21.15.62:0/2632810331 conn(0x7fca4c0d0d70 msgr2=0x7fca4c075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:04.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.571+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 shutdown_connections 2024-04-26T22:28:04.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.571+0000 7fca52472640 1 -- 172.21.15.62:0/2632810331 wait complete. 2024-04-26T22:28:04.584 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T22:28:04.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T22:28:04.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.694+0000 7f8f6c8cf640 1 Processor -- start 2024-04-26T22:28:04.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.694+0000 7f8f6c8cf640 1 -- start start 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f6c8cf640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8f68160800 0x7f8f681610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f6c8cf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f680d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f6c8cf640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f680d0930 con 0x7f8f680d2150 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f6c8cf640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8f680d2590 con 0x7f8f68160800 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f6c8cf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680d0c00 con 0x7f8f680d5b60 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f66d9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8f680d2150 0x7f8f680d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42340/0 (socket says 172.21.15.62:42340) 2024-04-26T22:28:04.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.695+0000 7f8f66d9d640 1 -- 172.21.15.62:0/3008443368 learned_addr learned my addr 172.21.15.62:0/3008443368 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f65d9b640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f680d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6659c640 1 --2- 172.21.15.62:0/3008443368 >> v2:172.21.15.174:3300/0 conn(0x7f8f68160800 0x7f8f681610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933999123 0 0) 0x7f8f680d0930 con 0x7f8f680d2150 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f54003520 con 0x7f8f680d2150 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 315719092 0 0) 0x7f8f54003520 con 0x7f8f680d2150 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 msgr2=0x7f8f680d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f680d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 >> v2:172.21.15.174:3300/0 conn(0x7f8f68160800 msgr2=0x7f8f681610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 --2- 172.21.15.62:0/3008443368 >> v2:172.21.15.174:3300/0 conn(0x7f8f68160800 0x7f8f681610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f680d0ee0 con 0x7f8f680d2150 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f8f580019f0 con 0x7f8f680d2150 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 >> v1:172.21.15.62:6789/0 conn(0x7f8f680d2150 legacy=0x7f8f680d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f54003f30 0x7f8f54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d2590 con 0x7f8f54004d40 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d0c00 con 0x7f8f54003f30 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5400b9c0 con 0x7f8f680d5b60 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f65d9b640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f54003f30 0x7f8f54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f6659c640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 -- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 msgr2=0x7f8f540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680d5b60 0x7f8f540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 -- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f54003f30 msgr2=0x7f8f54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f54003f30 0x7f8f54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.696+0000 7f8f66d9d640 1 -- 172.21.15.62:0/3008443368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f54003520 con 0x7f8f54004d40 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.697+0000 7f8f66d9d640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f540051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800a870 tx=0x7f8f54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710bec3bc756b98c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:04.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.697+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800a4c0 con 0x7f8f54004d40 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.697+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5802f070 con 0x7f8f54004d40 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.697+0000 7f8f6559a640 1 -- 172.21.15.62:0/3008443368 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5802bcb0 con 0x7f8f54004d40 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/3008443368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 msgr2=0x7f8f540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/3008443368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f540051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800a870 tx=0x7f8f54005830 comp rx=0 tx=0).stop 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f66d9d640 1 -- 172.21.15.62:0/3008443368 reap_dead start 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/3008443368 shutdown_connections 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/3008443368 >> 172.21.15.62:0/3008443368 conn(0x7f8f6809e550 msgr2=0x7f8f680d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/3008443368 shutdown_connections 2024-04-26T22:28:04.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.698+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/3008443368 wait complete. 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.699+0000 7f8f6c8cf640 1 Processor -- start 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.699+0000 7f8f6c8cf640 1 -- start start 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f68159590 0x7f8f680b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680b84f0 0x7f8f6815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d30f0 con 0x7f8f54004d40 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680d2e10 con 0x7f8f68159590 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6c8cf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680b6300 con 0x7f8f680b84f0 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43308/0 (socket says 172.21.15.62:43308) 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.700+0000 7f8f6659c640 1 -- 172.21.15.62:0/2777696582 learned_addr learned my addr 172.21.15.62:0/2777696582 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 -- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680b84f0 msgr2=0x7f8f6815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.704 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f66d9d640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680b84f0 0x7f8f6815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f65d9b640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f68159590 0x7f8f680b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680b84f0 0x7f8f6815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 -- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f68159590 msgr2=0x7f8f680b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f68159590 0x7f8f680b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 -- 172.21.15.62:0/2777696582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f68159b10 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6659c640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011a70 tx=0x7f8f60011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000fb50 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6000fcf0 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f68159df0 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000fea0 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.701+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f68157cd0 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.702+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6806a360 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.703+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f60012040 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.703+0000 7f8f5effd640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.703+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8f6008c1d0 con 0x7f8f54004d40 2024-04-26T22:28:04.705 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.703+0000 7f8f65d9b640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:04.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.704+0000 7f8f65d9b640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 0x7f8f4806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f681584d0 tx=0x7f8f50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:04.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.708+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f60019030 con 0x7f8f54004d40 2024-04-26T22:28:04.931 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:04.931+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8f680d2ba0 con 0x7f8f54004d40 2024-04-26T22:28:05.568 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.568+0000 7f8f5effd640 1 -- 172.21.15.62:0/2777696582 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f8f6005a690 con 0x7f8f54004d40 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 msgr2=0x7f8f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 0x7f8f4806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f681584d0 tx=0x7f8f50008040 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 msgr2=0x7f8f68157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f60011a70 tx=0x7f8f60011f30 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 shutdown_connections 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f680b84f0 0x7f8f6815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f68159590 0x7f8f680b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 --2- 172.21.15.62:0/2777696582 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f54004d40 0x7f8f68157790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 >> 172.21.15.62:0/2777696582 conn(0x7f8f6809e550 msgr2=0x7f8f680b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:05.571 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 shutdown_connections 2024-04-26T22:28:05.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.570+0000 7f8f6c8cf640 1 -- 172.21.15.62:0/2777696582 wait complete. 2024-04-26T22:28:05.583 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T22:28:05.583 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:05.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.689+0000 7f04aaf78640 1 Processor -- start 2024-04-26T22:28:05.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.689+0000 7f04aaf78640 1 -- start start 2024-04-26T22:28:05.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f04a415f080 0x7f04a406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f04a40ca8e0 0x7f04a40cad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f04a40cb290 con 0x7f04a40ca8e0 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f04a406d9c0 con 0x7f04a415f080 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04aaf78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f04a406c3e0 con 0x7f04a415f460 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04a9515640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f04a40ca8e0 0x7f04a40cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04a9515640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f04a40ca8e0 0x7f04a40cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43312/0 (socket says 172.21.15.62:43312) 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04a9515640 1 -- 172.21.15.62:0/221264599 learned_addr learned my addr 172.21.15.62:0/221264599 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04a8d14640 1 --2- 172.21.15.62:0/221264599 >> v2:172.21.15.174:3300/0 conn(0x7f04a415f080 0x7f04a406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.690+0000 7f04a3fff640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 -- 172.21.15.62:0/221264599 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 msgr2=0x7f04a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 -- 172.21.15.62:0/221264599 >> v2:172.21.15.174:3300/0 conn(0x7f04a415f080 msgr2=0x7f04a406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:05.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 --2- 172.21.15.62:0/221264599 >> v2:172.21.15.174:3300/0 conn(0x7f04a415f080 0x7f04a406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 -- 172.21.15.62:0/221264599 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a406c6c0 con 0x7f04a40ca8e0 2024-04-26T22:28:05.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a9515640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a40cad50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04a406e770 tx=0x7f049402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377d90af4b8c795 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:05.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.691+0000 7f04a37fe640 1 -- 172.21.15.62:0/221264599 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494037070 con 0x7f04a40ca8e0 2024-04-26T22:28:05.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04a37fe640 1 -- 172.21.15.62:0/221264599 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0494032440 con 0x7f04a40ca8e0 2024-04-26T22:28:05.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04a37fe640 1 -- 172.21.15.62:0/221264599 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494032810 con 0x7f04a40ca8e0 2024-04-26T22:28:05.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 -- 172.21.15.62:0/221264599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 msgr2=0x7f04a40cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a40cad50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04a406e770 tx=0x7f049402a210 comp rx=0 tx=0).stop 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 -- 172.21.15.62:0/221264599 shutdown_connections 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a40cad50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 --2- 172.21.15.62:0/221264599 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 --2- 172.21.15.62:0/221264599 >> v2:172.21.15.174:3300/0 conn(0x7f04a415f080 0x7f04a406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 -- 172.21.15.62:0/221264599 >> 172.21.15.62:0/221264599 conn(0x7f04a40ce050 msgr2=0x7f04a40ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 -- 172.21.15.62:0/221264599 shutdown_connections 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.692+0000 7f04aaf78640 1 -- 172.21.15.62:0/221264599 wait complete. 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.693+0000 7f04aaf78640 1 Processor -- start 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.693+0000 7f04aaf78640 1 -- start start 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a415f080 0x7f04a406fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f04a406cf60 con 0x7f04a40ca8e0 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04a406cc80 con 0x7f04a415f080 2024-04-26T22:28:05.697 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04aaf78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f04a40cdd00 con 0x7f04a415f460 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04a8d14640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04a8d14640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43326/0 (socket says 172.21.15.62:43326) 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04a8d14640 1 -- 172.21.15.62:0/1755142537 learned_addr learned my addr 172.21.15.62:0/1755142537 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04a9515640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.694+0000 7f04a3fff640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a415f080 0x7f04a406fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 -- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 msgr2=0x7f04a41599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a41599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 -- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a415f080 msgr2=0x7f04a406fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a415f080 0x7f04a406fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 -- 172.21.15.62:0/1755142537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a4070550 con 0x7f04a40ca8e0 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a8d14640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f049c007ef0 tx=0x7f049c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:05.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c014620 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a4070830 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049c019070 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c014a00 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.695+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a4153880 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.696+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a415b300 con 0x7f04a40ca8e0 2024-04-26T22:28:05.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.696+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f049c014ba0 con 0x7f04a40ca8e0 2024-04-26T22:28:05.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.697+0000 7f04a17fa640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 0x7f049006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:05.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.697+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f049c08c3c0 con 0x7f04a40ca8e0 2024-04-26T22:28:05.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.697+0000 7f04a3fff640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 0x7f049006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:05.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.698+0000 7f04a3fff640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 0x7f049006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04a4158b70 tx=0x7f0498008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:05.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.701+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f049c05a120 con 0x7f04a40ca8e0 2024-04-26T22:28:05.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.923+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04a406a360 con 0x7f04a40ca8e0 2024-04-26T22:28:05.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.923+0000 7f04a17fa640 1 -- 172.21.15.62:0/1755142537 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7f049c05e420 con 0x7f04a40ca8e0 2024-04-26T22:28:05.924 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:05.924 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:05.924 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.925+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 msgr2=0x7f049006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.925+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 0x7f049006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04a4158b70 tx=0x7f0498008040 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.925+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 msgr2=0x7f04a414ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.925+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f049c007ef0 tx=0x7f049c00e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 shutdown_connections 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f049006c8f0 0x7f049006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04a415f460 0x7f04a41599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a415f080 0x7f04a406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 --2- 172.21.15.62:0/1755142537 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04a40ca8e0 0x7f04a414ba20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:05.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 >> 172.21.15.62:0/1755142537 conn(0x7f04a40ce050 msgr2=0x7f04a40b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 shutdown_connections 2024-04-26T22:28:05.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:05.926+0000 7f04aaf78640 1 -- 172.21.15.62:0/1755142537 wait complete. 2024-04-26T22:28:05.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:06.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.037+0000 7f2aa297f640 1 Processor -- start 2024-04-26T22:28:06.037 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.037+0000 7f2aa297f640 1 -- start start 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2aa297f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2aa297f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 0x7f2a9c0d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2aa297f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9c0d3de0 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2aa297f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0d43a0 con 0x7f2a9c15f380 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2aa297f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0d40c0 con 0x7f2a9c15f760 2024-04-26T22:28:06.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43328/0 (socket says 172.21.15.62:43328) 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 -- 172.21.15.62:0/969666872 learned_addr learned my addr 172.21.15.62:0/969666872 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 -- 172.21.15.62:0/969666872 >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 msgr2=0x7f2a9c0d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9b7fe640 1 --2- 172.21.15.62:0/969666872 >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 0x7f2a9c0d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 --2- 172.21.15.62:0/969666872 >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 0x7f2a9c0d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 -- 172.21.15.62:0/969666872 >> v1:172.21.15.174:6789/0 conn(0x7f2a9c0d5b50 legacy=0x7f2a9c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9bfff640 1 -- 172.21.15.62:0/969666872 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0b8280 con 0x7f2a9c15f380 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.038+0000 7f2a9b7fe640 1 --2- 172.21.15.62:0/969666872 >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 0x7f2a9c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.039+0000 7f2a9bfff640 1 --2- 172.21.15.62:0/969666872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0d4520 tx=0x7f2a8402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd783fe3edae67f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.039+0000 7f2a9affd640 1 -- 172.21.15.62:0/969666872 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037070 con 0x7f2a9c15f380 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.039+0000 7f2a9affd640 1 -- 172.21.15.62:0/969666872 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84032440 con 0x7f2a9c15f380 2024-04-26T22:28:06.040 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.039+0000 7f2a9affd640 1 -- 172.21.15.62:0/969666872 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84032810 con 0x7f2a9c15f380 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 -- 172.21.15.62:0/969666872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c15f380 msgr2=0x7f2a9c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 --2- 172.21.15.62:0/969666872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0d4520 tx=0x7f2a8402a2c0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 -- 172.21.15.62:0/969666872 shutdown_connections 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 --2- 172.21.15.62:0/969666872 >> v2:172.21.15.174:3300/0 conn(0x7f2a9c15f760 0x7f2a9c0d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 --2- 172.21.15.62:0/969666872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c15f380 0x7f2a9c0d1bd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 -- 172.21.15.62:0/969666872 >> 172.21.15.62:0/969666872 conn(0x7f2a9c09e560 msgr2=0x7f2a9c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 -- 172.21.15.62:0/969666872 shutdown_connections 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.040+0000 7f2aa297f640 1 -- 172.21.15.62:0/969666872 wait complete. 2024-04-26T22:28:06.043 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.041+0000 7f2aa297f640 1 Processor -- start 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.041+0000 7f2aa297f640 1 -- start start 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c15f380 0x7f2a9c150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a9c15f760 0x7f2a9c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0d1280 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0d1100 con 0x7f2a9c15f380 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa297f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c0b9a90 con 0x7f2a9c15f760 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43336/0 (socket says 172.21.15.62:43336) 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 -- 172.21.15.62:0/1992331180 learned_addr learned my addr 172.21.15.62:0/1992331180 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2aa0f1c640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a9c15f760 0x7f2a9c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9b7fe640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c15f380 0x7f2a9c150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.044 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 -- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a9c15f760 msgr2=0x7f2a9c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a9c15f760 0x7f2a9c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 -- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c15f380 msgr2=0x7f2a9c150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c15f380 0x7f2a9c150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 -- 172.21.15.62:0/1992331180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c150a20 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a9bfff640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.042+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84041020 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.043+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84030970 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.043+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c150d00 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.043+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037040 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.043+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1512c0 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.044+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c06a360 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.044+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a84044020 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.044+0000 7f2a98ff9640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 0x7f2a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.044+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2a840ac5e0 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.045+0000 7f2a9b7fe640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 0x7f2a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.045+0000 7f2a9b7fe640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 0x7f2a7c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c1617c0 tx=0x7f2a8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.048 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.049+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a8404b030 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.234+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a9c0cf760 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.235+0000 7f2a98ff9640 1 -- 172.21.15.62:0/1992331180 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7f2a8407aa10 con 0x7f2a9c0d5b50 2024-04-26T22:28:06.235 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:06.235 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 msgr2=0x7f2a7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 0x7f2a7c06edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c1617c0 tx=0x7f2a8c008040 comp rx=0 tx=0).stop 2024-04-26T22:28:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 msgr2=0x7f2a9c0b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a840046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 shutdown_connections 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2a7c06c8f0 0x7f2a7c06edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a9c15f760 0x7f2a9c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9c15f380 0x7f2a9c150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 --2- 172.21.15.62:0/1992331180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2a9c0d5b50 0x7f2a9c0b55b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 >> 172.21.15.62:0/1992331180 conn(0x7f2a9c09e560 msgr2=0x7f2a9c0b7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 shutdown_connections 2024-04-26T22:28:06.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.237+0000 7f2aa297f640 1 -- 172.21.15.62:0/1992331180 wait complete. 2024-04-26T22:28:06.248 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:28:06.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:06.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 Processor -- start 2024-04-26T22:28:06.349 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 -- start start 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0e80162b40 0x7f0e800d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e800d3820 0x7f0e800d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e800d0210 con 0x7f0e800d1850 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800d41c0 con 0x7f0e800d3820 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.349+0000 7f0e85fec640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800d04f0 con 0x7f0e80162b40 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e800d3820 0x7f0e800d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0e800d3820 0x7f0e800d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43342/0 (socket says 172.21.15.62:43342) 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 -- 172.21.15.62:0/3651044573 learned_addr learned my addr 172.21.15.62:0/3651044573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7f7fe640 1 --2- 172.21.15.62:0/3651044573 >> v2:172.21.15.174:3300/0 conn(0x7f0e80162b40 0x7f0e800d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 -- 172.21.15.62:0/3651044573 >> v2:172.21.15.174:3300/0 conn(0x7f0e80162b40 msgr2=0x7f0e800d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 --2- 172.21.15.62:0/3651044573 >> v2:172.21.15.174:3300/0 conn(0x7f0e80162b40 0x7f0e800d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 -- 172.21.15.62:0/3651044573 >> v1:172.21.15.174:6789/0 conn(0x7f0e800d1850 legacy=0x7f0e800d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.351 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.350+0000 7f0e7effd640 1 -- 172.21.15.62:0/3651044573 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800d07d0 con 0x7f0e800d3820 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e7effd640 1 --2- 172.21.15.62:0/3651044573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e800d3820 0x7f0e800d3c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0e800d2910 tx=0x7f0e6802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66381fd6c485a899 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e7e7fc640 1 -- 172.21.15.62:0/3651044573 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68037070 con 0x7f0e800d3820 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e7e7fc640 1 -- 172.21.15.62:0/3651044573 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6802ae50 con 0x7f0e800d3820 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e7e7fc640 1 -- 172.21.15.62:0/3651044573 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68032860 con 0x7f0e800d3820 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 -- 172.21.15.62:0/3651044573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e800d3820 msgr2=0x7f0e800d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 --2- 172.21.15.62:0/3651044573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e800d3820 0x7f0e800d3c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0e800d2910 tx=0x7f0e6802a300 comp rx=0 tx=0).stop 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 -- 172.21.15.62:0/3651044573 shutdown_connections 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 --2- 172.21.15.62:0/3651044573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e800d3820 0x7f0e800d3c80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.352 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 --2- 172.21.15.62:0/3651044573 >> v2:172.21.15.174:3300/0 conn(0x7f0e80162b40 0x7f0e800d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 -- 172.21.15.62:0/3651044573 >> 172.21.15.62:0/3651044573 conn(0x7f0e8009e550 msgr2=0x7f0e800d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 -- 172.21.15.62:0/3651044573 shutdown_connections 2024-04-26T22:28:06.353 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.351+0000 7f0e85fec640 1 -- 172.21.15.62:0/3651044573 wait complete. 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.352+0000 7f0e85fec640 1 Processor -- start 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 -- start start 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e800d1850 0x7f0e800caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 0x7f0e800b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800b5bb0 con 0x7f0e80162b40 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e800d0d90 con 0x7f0e800d3820 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e85fec640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0e800d6110 con 0x7f0e800d1850 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.353+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43348/0 (socket says 172.21.15.62:43348) 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 -- 172.21.15.62:0/929207912 learned_addr learned my addr 172.21.15.62:0/929207912 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7f7fe640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e800d1850 0x7f0e800caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7effd640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 0x7f0e800b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.355 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 -- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e800d1850 msgr2=0x7f0e800caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e800d1850 0x7f0e800caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 -- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 msgr2=0x7f0e800b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 0x7f0e800b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 -- 172.21.15.62:0/929207912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e800cb000 con 0x7f0e80162b40 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7effd640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 0x7f0e800b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e7ffff640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c760 tx=0x7f0e7800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78016020 con 0x7f0e80162b40 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e78006d50 con 0x7f0e80162b40 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e800b8360 con 0x7f0e80162b40 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78003bb0 con 0x7f0e80162b40 2024-04-26T22:28:06.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.354+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e800b8920 con 0x7f0e80162b40 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.355+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8006a360 con 0x7f0e80162b40 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.356+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e780050d0 con 0x7f0e80162b40 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.356+0000 7f0e77fff640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.356+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0e7808b870 con 0x7f0e80162b40 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.356+0000 7f0e7f7fe640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.357 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.357+0000 7f0e7f7fe640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009ad0 tx=0x7f0e70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.360 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.360+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e78020020 con 0x7f0e80162b40 2024-04-26T22:28:06.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.543+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e800b7810 con 0x7f0e6006c920 2024-04-26T22:28:06.549 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.549+0000 7f0e77fff640 1 -- 172.21.15.62:0/929207912 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7f0e800b7810 con 0x7f0e6006c920 2024-04-26T22:28:06.552 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:06.553 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 msgr2=0x7f0e6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 0x7f0e6006ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009ad0 tx=0x7f0e70008040 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 msgr2=0x7f0e800b9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c760 tx=0x7f0e7800cb60 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 shutdown_connections 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0e6006c920 0x7f0e6006ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0e80162b40 0x7f0e800b9400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e800d3820 0x7f0e800b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 --2- 172.21.15.62:0/929207912 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0e800d1850 0x7f0e800caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 >> 172.21.15.62:0/929207912 conn(0x7f0e8009e550 msgr2=0x7f0e800b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 shutdown_connections 2024-04-26T22:28:06.555 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.553+0000 7f0e85fec640 1 -- 172.21.15.62:0/929207912 wait complete. 2024-04-26T22:28:06.564 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-26T22:28:05.651527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2109795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.120179"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544043+0000","last_change":"2024-04-26T22:28:01.544043+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544043+0000","last_undegraded":"2024-04-26T22:28:01.544043+0000","last_fullsized":"2024-04-26T22:28:01.544043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.555306+0000","last_change":"2024-04-26T22:28:01.555306+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.555306+0000","last_undegraded":"2024-04-26T22:28:01.555306+0000","last_fullsized":"2024-04-26T22:28:01.555306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540215+0000","last_change":"2024-04-26T22:28:01.540215+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540215+0000","last_undegraded":"2024-04-26T22:28:01.540215+0000","last_fullsized":"2024-04-26T22:28:01.540215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550106+0000","last_change":"2024-04-26T22:28:01.550106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550106+0000","last_undegraded":"2024-04-26T22:28:01.550106+0000","last_fullsized":"2024-04-26T22:28:01.550106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544326+0000","last_change":"2024-04-26T22:28:01.544326+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544326+0000","last_undegraded":"2024-04-26T22:28:01.544326+0000","last_fullsized":"2024-04-26T22:28:01.544326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554580+0000","last_change":"2024-04-26T22:28:01.554580+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554580+0000","last_undegraded":"2024-04-26T22:28:01.554580+0000","last_fullsized":"2024-04-26T22:28:01.554580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547702+0000","last_change":"2024-04-26T22:28:01.547702+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547702+0000","last_undegraded":"2024-04-26T22:28:01.547702+0000","last_fullsized":"2024-04-26T22:28:01.547702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550612+0000","last_change":"2024-04-26T22:28:01.550612+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550612+0000","last_undegraded":"2024-04-26T22:28:01.550612+0000","last_fullsized":"2024-04-26T22:28:01.550612+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541137+0000","last_change":"2024-04-26T22:28:01.541137+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541137+0000","last_undegraded":"2024-04-26T22:28:01.541137+0000","last_fullsized":"2024-04-26T22:28:01.541137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550090+0000","last_change":"2024-04-26T22:28:01.550090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550090+0000","last_undegraded":"2024-04-26T22:28:01.550090+0000","last_fullsized":"2024-04-26T22:28:01.550090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546936+0000","last_change":"2024-04-26T22:28:01.546936+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546936+0000","last_undegraded":"2024-04-26T22:28:01.546936+0000","last_fullsized":"2024-04-26T22:28:01.546936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543309+0000","last_change":"2024-04-26T22:28:01.543309+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543309+0000","last_undegraded":"2024-04-26T22:28:01.543309+0000","last_fullsized":"2024-04-26T22:28:01.543309+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543892+0000","last_change":"2024-04-26T22:28:01.543892+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543892+0000","last_undegraded":"2024-04-26T22:28:01.543892+0000","last_fullsized":"2024-04-26T22:28:01.543892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554052+0000","last_change":"2024-04-26T22:28:01.554052+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554052+0000","last_undegraded":"2024-04-26T22:28:01.554052+0000","last_fullsized":"2024-04-26T22:28:01.554052+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540104+0000","last_change":"2024-04-26T22:28:01.540104+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540104+0000","last_undegraded":"2024-04-26T22:28:01.540104+0000","last_fullsized":"2024-04-26T22:28:01.540104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546355+0000","last_change":"2024-04-26T22:28:01.546355+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546355+0000","last_undegraded":"2024-04-26T22:28:01.546355+0000","last_fullsized":"2024-04-26T22:28:01.546355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546942+0000","last_change":"2024-04-26T22:28:01.546942+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546942+0000","last_undegraded":"2024-04-26T22:28:01.546942+0000","last_fullsized":"2024-04-26T22:28:01.546942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552774+0000","last_change":"2024-04-26T22:28:01.552774+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552774+0000","last_undegraded":"2024-04-26T22:28:01.552774+0000","last_fullsized":"2024-04-26T22:28:01.552774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552113+0000","last_change":"2024-04-26T22:28:01.552113+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552113+0000","last_undegraded":"2024-04-26T22:28:01.552113+0000","last_fullsized":"2024-04-26T22:28:01.552113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542939+0000","last_change":"2024-04-26T22:28:01.542939+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542939+0000","last_undegraded":"2024-04-26T22:28:01.542939+0000","last_fullsized":"2024-04-26T22:28:01.542939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543242+0000","last_change":"2024-04-26T22:28:01.543242+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543242+0000","last_undegraded":"2024-04-26T22:28:01.543242+0000","last_fullsized":"2024-04-26T22:28:01.543242+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540859+0000","last_change":"2024-04-26T22:28:01.540859+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540859+0000","last_undegraded":"2024-04-26T22:28:01.540859+0000","last_fullsized":"2024-04-26T22:28:01.540859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546687+0000","last_change":"2024-04-26T22:28:01.546687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546687+0000","last_undegraded":"2024-04-26T22:28:01.546687+0000","last_fullsized":"2024-04-26T22:28:01.546687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540745+0000","last_change":"2024-04-26T22:28:01.540745+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540745+0000","last_undegraded":"2024-04-26T22:28:01.540745+0000","last_fullsized":"2024-04-26T22:28:01.540745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550111+0000","last_change":"2024-04-26T22:28:01.550111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550111+0000","last_undegraded":"2024-04-26T22:28:01.550111+0000","last_fullsized":"2024-04-26T22:28:01.550111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551624+0000","last_change":"2024-04-26T22:28:01.551624+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551624+0000","last_undegraded":"2024-04-26T22:28:01.551624+0000","last_fullsized":"2024-04-26T22:28:01.551624+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550886+0000","last_change":"2024-04-26T22:28:01.550886+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550886+0000","last_undegraded":"2024-04-26T22:28:01.550886+0000","last_fullsized":"2024-04-26T22:28:01.550886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540303+0000","last_change":"2024-04-26T22:28:01.540303+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540303+0000","last_undegraded":"2024-04-26T22:28:01.540303+0000","last_fullsized":"2024-04-26T22:28:01.540303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541775+0000","last_change":"2024-04-26T22:28:01.541775+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541775+0000","last_undegraded":"2024-04-26T22:28:01.541775+0000","last_fullsized":"2024-04-26T22:28:01.541775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546895+0000","last_change":"2024-04-26T22:28:01.546895+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546895+0000","last_undegraded":"2024-04-26T22:28:01.546895+0000","last_fullsized":"2024-04-26T22:28:01.546895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543109+0000","last_change":"2024-04-26T22:28:01.543109+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543109+0000","last_undegraded":"2024-04-26T22:28:01.543109+0000","last_fullsized":"2024-04-26T22:28:01.543109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550216+0000","last_change":"2024-04-26T22:28:01.550216+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550216+0000","last_undegraded":"2024-04-26T22:28:01.550216+0000","last_fullsized":"2024-04-26T22:28:01.550216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547102+0000","last_change":"2024-04-26T22:28:01.547102+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547102+0000","last_undegraded":"2024-04-26T22:28:01.547102+0000","last_fullsized":"2024-04-26T22:28:01.547102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554323+0000","last_change":"2024-04-26T22:28:01.551148+0000","last_active":"2024-04-26T22:28:01.554323+0000","last_peered":"2024-04-26T22:28:01.554323+0000","last_clean":"2024-04-26T22:28:01.554323+0000","last_became_active":"2024-04-26T22:28:01.550854+0000","last_became_peered":"2024-04-26T22:28:01.550854+0000","last_unstale":"2024-04-26T22:28:01.554323+0000","last_undegraded":"2024-04-26T22:28:01.554323+0000","last_fullsized":"2024-04-26T22:28:01.554323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543631+0000","last_change":"2024-04-26T22:28:01.543631+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543631+0000","last_undegraded":"2024-04-26T22:28:01.543631+0000","last_fullsized":"2024-04-26T22:28:01.543631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556848+0000","last_change":"2024-04-26T22:28:01.553286+0000","last_active":"2024-04-26T22:28:01.556848+0000","last_peered":"2024-04-26T22:28:01.556848+0000","last_clean":"2024-04-26T22:28:01.556848+0000","last_became_active":"2024-04-26T22:28:01.552926+0000","last_became_peered":"2024-04-26T22:28:01.552926+0000","last_unstale":"2024-04-26T22:28:01.556848+0000","last_undegraded":"2024-04-26T22:28:01.556848+0000","last_fullsized":"2024-04-26T22:28:01.556848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541804+0000","last_change":"2024-04-26T22:28:01.541804+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541804+0000","last_undegraded":"2024-04-26T22:28:01.541804+0000","last_fullsized":"2024-04-26T22:28:01.541804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566207+0000","last_change":"2024-04-26T22:28:01.559844+0000","last_active":"2024-04-26T22:28:01.566207+0000","last_peered":"2024-04-26T22:28:01.566207+0000","last_clean":"2024-04-26T22:28:01.566207+0000","last_became_active":"2024-04-26T22:28:01.559546+0000","last_became_peered":"2024-04-26T22:28:01.559546+0000","last_unstale":"2024-04-26T22:28:01.566207+0000","last_undegraded":"2024-04-26T22:28:01.566207+0000","last_fullsized":"2024-04-26T22:28:01.566207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:59.038151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550464+0000","last_change":"2024-04-26T22:28:01.550464+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550464+0000","last_undegraded":"2024-04-26T22:28:01.550464+0000","last_fullsized":"2024-04-26T22:28:01.550464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554674+0000","last_change":"2024-04-26T22:28:01.552326+0000","last_active":"2024-04-26T22:28:01.554674+0000","last_peered":"2024-04-26T22:28:01.554674+0000","last_clean":"2024-04-26T22:28:01.554674+0000","last_became_active":"2024-04-26T22:28:01.551423+0000","last_became_peered":"2024-04-26T22:28:01.551423+0000","last_unstale":"2024-04-26T22:28:01.554674+0000","last_undegraded":"2024-04-26T22:28:01.554674+0000","last_fullsized":"2024-04-26T22:28:01.554674+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:17.520923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546338+0000","last_change":"2024-04-26T22:28:01.546338+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546338+0000","last_undegraded":"2024-04-26T22:28:01.546338+0000","last_fullsized":"2024-04-26T22:28:01.546338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554857+0000","last_change":"2024-04-26T22:28:01.549390+0000","last_active":"2024-04-26T22:28:01.554857+0000","last_peered":"2024-04-26T22:28:01.554857+0000","last_clean":"2024-04-26T22:28:01.554857+0000","last_became_active":"2024-04-26T22:28:01.548063+0000","last_became_peered":"2024-04-26T22:28:01.548063+0000","last_unstale":"2024-04-26T22:28:01.554857+0000","last_undegraded":"2024-04-26T22:28:01.554857+0000","last_fullsized":"2024-04-26T22:28:01.554857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.079402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540460+0000","last_change":"2024-04-26T22:28:01.540460+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540460+0000","last_undegraded":"2024-04-26T22:28:01.540460+0000","last_fullsized":"2024-04-26T22:28:01.540460+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553031+0000","last_change":"2024-04-26T22:28:01.548404+0000","last_active":"2024-04-26T22:28:01.553031+0000","last_peered":"2024-04-26T22:28:01.553031+0000","last_clean":"2024-04-26T22:28:01.553031+0000","last_became_active":"2024-04-26T22:28:01.548088+0000","last_became_peered":"2024-04-26T22:28:01.548088+0000","last_unstale":"2024-04-26T22:28:01.553031+0000","last_undegraded":"2024-04-26T22:28:01.553031+0000","last_fullsized":"2024-04-26T22:28:01.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:38.650628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542687+0000","last_change":"2024-04-26T22:28:01.542687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542687+0000","last_undegraded":"2024-04-26T22:28:01.542687+0000","last_fullsized":"2024-04-26T22:28:01.542687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563282+0000","last_change":"2024-04-26T22:28:01.556939+0000","last_active":"2024-04-26T22:28:01.563282+0000","last_peered":"2024-04-26T22:28:01.563282+0000","last_clean":"2024-04-26T22:28:01.563282+0000","last_became_active":"2024-04-26T22:28:01.556663+0000","last_became_peered":"2024-04-26T22:28:01.556663+0000","last_unstale":"2024-04-26T22:28:01.563282+0000","last_undegraded":"2024-04-26T22:28:01.563282+0000","last_fullsized":"2024-04-26T22:28:01.563282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549555+0000","last_change":"2024-04-26T22:28:01.549555+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549555+0000","last_undegraded":"2024-04-26T22:28:01.549555+0000","last_fullsized":"2024-04-26T22:28:01.549555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554932+0000","last_change":"2024-04-26T22:28:01.552353+0000","last_active":"2024-04-26T22:28:01.554932+0000","last_peered":"2024-04-26T22:28:01.554932+0000","last_clean":"2024-04-26T22:28:01.554932+0000","last_became_active":"2024-04-26T22:28:01.549925+0000","last_became_peered":"2024-04-26T22:28:01.549925+0000","last_unstale":"2024-04-26T22:28:01.554932+0000","last_undegraded":"2024-04-26T22:28:01.554932+0000","last_fullsized":"2024-04-26T22:28:01.554932+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:42.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543786+0000","last_change":"2024-04-26T22:28:01.543786+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543786+0000","last_undegraded":"2024-04-26T22:28:01.543786+0000","last_fullsized":"2024-04-26T22:28:01.543786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558468+0000","last_change":"2024-04-26T22:28:01.555381+0000","last_active":"2024-04-26T22:28:01.558468+0000","last_peered":"2024-04-26T22:28:01.558468+0000","last_clean":"2024-04-26T22:28:01.558468+0000","last_became_active":"2024-04-26T22:28:01.555132+0000","last_became_peered":"2024-04-26T22:28:01.555132+0000","last_unstale":"2024-04-26T22:28:01.558468+0000","last_undegraded":"2024-04-26T22:28:01.558468+0000","last_fullsized":"2024-04-26T22:28:01.558468+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.553709+0000","last_change":"2024-04-26T22:28:01.553709+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.553709+0000","last_undegraded":"2024-04-26T22:28:01.553709+0000","last_fullsized":"2024-04-26T22:28:01.553709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566486+0000","last_change":"2024-04-26T22:28:01.563856+0000","last_active":"2024-04-26T22:28:01.566486+0000","last_peered":"2024-04-26T22:28:01.566486+0000","last_clean":"2024-04-26T22:28:01.566486+0000","last_became_active":"2024-04-26T22:28:01.563514+0000","last_became_peered":"2024-04-26T22:28:01.563514+0000","last_unstale":"2024-04-26T22:28:01.566486+0000","last_undegraded":"2024-04-26T22:28:01.566486+0000","last_fullsized":"2024-04-26T22:28:01.566486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551090+0000","last_change":"2024-04-26T22:28:01.551090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551090+0000","last_undegraded":"2024-04-26T22:28:01.551090+0000","last_fullsized":"2024-04-26T22:28:01.551090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555437+0000","last_change":"2024-04-26T22:28:01.553958+0000","last_active":"2024-04-26T22:28:01.555437+0000","last_peered":"2024-04-26T22:28:01.555437+0000","last_clean":"2024-04-26T22:28:01.555437+0000","last_became_active":"2024-04-26T22:28:01.553552+0000","last_became_peered":"2024-04-26T22:28:01.553552+0000","last_unstale":"2024-04-26T22:28:01.555437+0000","last_undegraded":"2024-04-26T22:28:01.555437+0000","last_fullsized":"2024-04-26T22:28:01.555437+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540973+0000","last_change":"2024-04-26T22:28:01.540973+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540973+0000","last_undegraded":"2024-04-26T22:28:01.540973+0000","last_fullsized":"2024-04-26T22:28:01.540973+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566167+0000","last_change":"2024-04-26T22:28:01.559144+0000","last_active":"2024-04-26T22:28:01.566167+0000","last_peered":"2024-04-26T22:28:01.566167+0000","last_clean":"2024-04-26T22:28:01.566167+0000","last_became_active":"2024-04-26T22:28:01.558496+0000","last_became_peered":"2024-04-26T22:28:01.558496+0000","last_unstale":"2024-04-26T22:28:01.566167+0000","last_undegraded":"2024-04-26T22:28:01.566167+0000","last_fullsized":"2024-04-26T22:28:01.566167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551099+0000","last_change":"2024-04-26T22:28:01.551099+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551099+0000","last_undegraded":"2024-04-26T22:28:01.551099+0000","last_fullsized":"2024-04-26T22:28:01.551099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554892+0000","last_change":"2024-04-26T22:28:01.550978+0000","last_active":"2024-04-26T22:28:01.554892+0000","last_peered":"2024-04-26T22:28:01.554892+0000","last_clean":"2024-04-26T22:28:01.554892+0000","last_became_active":"2024-04-26T22:28:01.550154+0000","last_became_peered":"2024-04-26T22:28:01.550154+0000","last_unstale":"2024-04-26T22:28:01.554892+0000","last_undegraded":"2024-04-26T22:28:01.554892+0000","last_fullsized":"2024-04-26T22:28:01.554892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:59.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550436+0000","last_change":"2024-04-26T22:28:01.550436+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550436+0000","last_undegraded":"2024-04-26T22:28:01.550436+0000","last_fullsized":"2024-04-26T22:28:01.550436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559137+0000","last_change":"2024-04-26T22:28:01.551899+0000","last_active":"2024-04-26T22:28:01.559137+0000","last_peered":"2024-04-26T22:28:01.559137+0000","last_clean":"2024-04-26T22:28:01.559137+0000","last_became_active":"2024-04-26T22:28:01.551587+0000","last_became_peered":"2024-04-26T22:28:01.551587+0000","last_unstale":"2024-04-26T22:28:01.559137+0000","last_undegraded":"2024-04-26T22:28:01.559137+0000","last_fullsized":"2024-04-26T22:28:01.559137+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:09.972664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549849+0000","last_change":"2024-04-26T22:28:01.549849+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549849+0000","last_undegraded":"2024-04-26T22:28:01.549849+0000","last_fullsized":"2024-04-26T22:28:01.549849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558181+0000","last_change":"2024-04-26T22:28:01.555910+0000","last_active":"2024-04-26T22:28:01.558181+0000","last_peered":"2024-04-26T22:28:01.558181+0000","last_clean":"2024-04-26T22:28:01.558181+0000","last_became_active":"2024-04-26T22:28:01.555596+0000","last_became_peered":"2024-04-26T22:28:01.555596+0000","last_unstale":"2024-04-26T22:28:01.558181+0000","last_undegraded":"2024-04-26T22:28:01.558181+0000","last_fullsized":"2024-04-26T22:28:01.558181+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542675+0000","last_change":"2024-04-26T22:28:01.542675+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542675+0000","last_undegraded":"2024-04-26T22:28:01.542675+0000","last_fullsized":"2024-04-26T22:28:01.542675+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566270+0000","last_change":"2024-04-26T22:28:01.559695+0000","last_active":"2024-04-26T22:28:01.566270+0000","last_peered":"2024-04-26T22:28:01.566270+0000","last_clean":"2024-04-26T22:28:01.566270+0000","last_became_active":"2024-04-26T22:28:01.559448+0000","last_became_peered":"2024-04-26T22:28:01.559448+0000","last_unstale":"2024-04-26T22:28:01.566270+0000","last_undegraded":"2024-04-26T22:28:01.566270+0000","last_fullsized":"2024-04-26T22:28:01.566270+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542254+0000","last_change":"2024-04-26T22:28:01.542254+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542254+0000","last_undegraded":"2024-04-26T22:28:01.542254+0000","last_fullsized":"2024-04-26T22:28:01.542254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566076+0000","last_change":"2024-04-26T22:28:01.560134+0000","last_active":"2024-04-26T22:28:01.566076+0000","last_peered":"2024-04-26T22:28:01.566076+0000","last_clean":"2024-04-26T22:28:01.566076+0000","last_became_active":"2024-04-26T22:28:01.559832+0000","last_became_peered":"2024-04-26T22:28:01.559832+0000","last_unstale":"2024-04-26T22:28:01.566076+0000","last_undegraded":"2024-04-26T22:28:01.566076+0000","last_fullsized":"2024-04-26T22:28:01.566076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542106+0000","last_change":"2024-04-26T22:28:01.542106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542106+0000","last_undegraded":"2024-04-26T22:28:01.542106+0000","last_fullsized":"2024-04-26T22:28:01.542106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554944+0000","last_change":"2024-04-26T22:28:01.550770+0000","last_active":"2024-04-26T22:28:01.554944+0000","last_peered":"2024-04-26T22:28:01.554944+0000","last_clean":"2024-04-26T22:28:01.554944+0000","last_became_active":"2024-04-26T22:28:01.550490+0000","last_became_peered":"2024-04-26T22:28:01.550490+0000","last_unstale":"2024-04-26T22:28:01.554944+0000","last_undegraded":"2024-04-26T22:28:01.554944+0000","last_fullsized":"2024-04-26T22:28:01.554944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:29.207017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549567+0000","last_change":"2024-04-26T22:28:01.549567+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549567+0000","last_undegraded":"2024-04-26T22:28:01.549567+0000","last_fullsized":"2024-04-26T22:28:01.549567+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553033+0000","last_change":"2024-04-26T22:28:01.548123+0000","last_active":"2024-04-26T22:28:01.553033+0000","last_peered":"2024-04-26T22:28:01.553033+0000","last_clean":"2024-04-26T22:28:01.553033+0000","last_became_active":"2024-04-26T22:28:01.547847+0000","last_became_peered":"2024-04-26T22:28:01.547847+0000","last_unstale":"2024-04-26T22:28:01.553033+0000","last_undegraded":"2024-04-26T22:28:01.553033+0000","last_fullsized":"2024-04-26T22:28:01.553033+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:50.283072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543141+0000","last_change":"2024-04-26T22:28:01.543141+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543141+0000","last_undegraded":"2024-04-26T22:28:01.543141+0000","last_fullsized":"2024-04-26T22:28:01.543141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554806+0000","last_change":"2024-04-26T22:28:01.548757+0000","last_active":"2024-04-26T22:28:01.554806+0000","last_peered":"2024-04-26T22:28:01.554806+0000","last_clean":"2024-04-26T22:28:01.554806+0000","last_became_active":"2024-04-26T22:28:01.548498+0000","last_became_peered":"2024-04-26T22:28:01.548498+0000","last_unstale":"2024-04-26T22:28:01.554806+0000","last_undegraded":"2024-04-26T22:28:01.554806+0000","last_fullsized":"2024-04-26T22:28:01.554806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.207881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543163+0000","last_change":"2024-04-26T22:28:01.543163+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543163+0000","last_undegraded":"2024-04-26T22:28:01.543163+0000","last_fullsized":"2024-04-26T22:28:01.543163+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.560904+0000","last_change":"2024-04-26T22:28:01.559890+0000","last_active":"2024-04-26T22:28:01.560904+0000","last_peered":"2024-04-26T22:28:01.560904+0000","last_clean":"2024-04-26T22:28:01.560904+0000","last_became_active":"2024-04-26T22:28:01.559564+0000","last_became_peered":"2024-04-26T22:28:01.559564+0000","last_unstale":"2024-04-26T22:28:01.560904+0000","last_undegraded":"2024-04-26T22:28:01.560904+0000","last_fullsized":"2024-04-26T22:28:01.560904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543175+0000","last_change":"2024-04-26T22:28:01.543175+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543175+0000","last_undegraded":"2024-04-26T22:28:01.543175+0000","last_fullsized":"2024-04-26T22:28:01.543175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559529+0000","last_change":"2024-04-26T22:28:01.558972+0000","last_active":"2024-04-26T22:28:01.559529+0000","last_peered":"2024-04-26T22:28:01.559529+0000","last_clean":"2024-04-26T22:28:01.559529+0000","last_became_active":"2024-04-26T22:28:01.558693+0000","last_became_peered":"2024-04-26T22:28:01.558693+0000","last_unstale":"2024-04-26T22:28:01.559529+0000","last_undegraded":"2024-04-26T22:28:01.559529+0000","last_fullsized":"2024-04-26T22:28:01.559529+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542997+0000","last_change":"2024-04-26T22:28:01.542997+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542997+0000","last_undegraded":"2024-04-26T22:28:01.542997+0000","last_fullsized":"2024-04-26T22:28:01.542997+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555059+0000","last_change":"2024-04-26T22:28:01.551281+0000","last_active":"2024-04-26T22:28:01.555059+0000","last_peered":"2024-04-26T22:28:01.555059+0000","last_clean":"2024-04-26T22:28:01.555059+0000","last_became_active":"2024-04-26T22:28:01.550195+0000","last_became_peered":"2024-04-26T22:28:01.550195+0000","last_unstale":"2024-04-26T22:28:01.555059+0000","last_undegraded":"2024-04-26T22:28:01.555059+0000","last_fullsized":"2024-04-26T22:28:01.555059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:58.163201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.548111+0000","last_change":"2024-04-26T22:28:01.548111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.548111+0000","last_undegraded":"2024-04-26T22:28:01.548111+0000","last_fullsized":"2024-04-26T22:28:01.548111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555367+0000","last_change":"2024-04-26T22:28:01.553831+0000","last_active":"2024-04-26T22:28:01.555367+0000","last_peered":"2024-04-26T22:28:01.555367+0000","last_clean":"2024-04-26T22:28:01.555367+0000","last_became_active":"2024-04-26T22:28:01.553563+0000","last_became_peered":"2024-04-26T22:28:01.553563+0000","last_unstale":"2024-04-26T22:28:01.555367+0000","last_undegraded":"2024-04-26T22:28:01.555367+0000","last_fullsized":"2024-04-26T22:28:01.555367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543247+0000","last_change":"2024-04-26T22:28:01.543247+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543247+0000","last_undegraded":"2024-04-26T22:28:01.543247+0000","last_fullsized":"2024-04-26T22:28:01.543247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565921+0000","last_change":"2024-04-26T22:28:01.558893+0000","last_active":"2024-04-26T22:28:01.565921+0000","last_peered":"2024-04-26T22:28:01.565921+0000","last_clean":"2024-04-26T22:28:01.565921+0000","last_became_active":"2024-04-26T22:28:01.558541+0000","last_became_peered":"2024-04-26T22:28:01.558541+0000","last_unstale":"2024-04-26T22:28:01.565921+0000","last_undegraded":"2024-04-26T22:28:01.565921+0000","last_fullsized":"2024-04-26T22:28:01.565921+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540152+0000","last_change":"2024-04-26T22:28:01.540152+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540152+0000","last_undegraded":"2024-04-26T22:28:01.540152+0000","last_fullsized":"2024-04-26T22:28:01.540152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563366+0000","last_change":"2024-04-26T22:28:01.560436+0000","last_active":"2024-04-26T22:28:01.563366+0000","last_peered":"2024-04-26T22:28:01.563366+0000","last_clean":"2024-04-26T22:28:01.563366+0000","last_became_active":"2024-04-26T22:28:01.560153+0000","last_became_peered":"2024-04-26T22:28:01.560153+0000","last_unstale":"2024-04-26T22:28:01.563366+0000","last_undegraded":"2024-04-26T22:28:01.563366+0000","last_fullsized":"2024-04-26T22:28:01.563366+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546676+0000","last_change":"2024-04-26T22:28:01.546676+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546676+0000","last_undegraded":"2024-04-26T22:28:01.546676+0000","last_fullsized":"2024-04-26T22:28:01.546676+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565905+0000","last_change":"2024-04-26T22:28:01.558609+0000","last_active":"2024-04-26T22:28:01.565905+0000","last_peered":"2024-04-26T22:28:01.565905+0000","last_clean":"2024-04-26T22:28:01.565905+0000","last_became_active":"2024-04-26T22:28:01.558332+0000","last_became_peered":"2024-04-26T22:28:01.558332+0000","last_unstale":"2024-04-26T22:28:01.565905+0000","last_undegraded":"2024-04-26T22:28:01.565905+0000","last_fullsized":"2024-04-26T22:28:01.565905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543092+0000","last_change":"2024-04-26T22:28:01.543092+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543092+0000","last_undegraded":"2024-04-26T22:28:01.543092+0000","last_fullsized":"2024-04-26T22:28:01.543092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556844+0000","last_change":"2024-04-26T22:28:01.551370+0000","last_active":"2024-04-26T22:28:01.556844+0000","last_peered":"2024-04-26T22:28:01.556844+0000","last_clean":"2024-04-26T22:28:01.556844+0000","last_became_active":"2024-04-26T22:28:01.550977+0000","last_became_peered":"2024-04-26T22:28:01.550977+0000","last_unstale":"2024-04-26T22:28:01.556844+0000","last_undegraded":"2024-04-26T22:28:01.556844+0000","last_fullsized":"2024-04-26T22:28:01.556844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546076+0000","last_change":"2024-04-26T22:28:01.546076+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546076+0000","last_undegraded":"2024-04-26T22:28:01.546076+0000","last_fullsized":"2024-04-26T22:28:01.546076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559355+0000","last_change":"2024-04-26T22:28:01.555714+0000","last_active":"2024-04-26T22:28:01.559355+0000","last_peered":"2024-04-26T22:28:01.559355+0000","last_clean":"2024-04-26T22:28:01.559355+0000","last_became_active":"2024-04-26T22:28:01.555437+0000","last_became_peered":"2024-04-26T22:28:01.555437+0000","last_unstale":"2024-04-26T22:28:01.559355+0000","last_undegraded":"2024-04-26T22:28:01.559355+0000","last_fullsized":"2024-04-26T22:28:01.559355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549204+0000","last_change":"2024-04-26T22:28:01.549204+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549204+0000","last_undegraded":"2024-04-26T22:28:01.549204+0000","last_fullsized":"2024-04-26T22:28:01.549204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554873+0000","last_change":"2024-04-26T22:28:01.549998+0000","last_active":"2024-04-26T22:28:01.554873+0000","last_peered":"2024-04-26T22:28:01.554873+0000","last_clean":"2024-04-26T22:28:01.554873+0000","last_became_active":"2024-04-26T22:28:01.549526+0000","last_became_peered":"2024-04-26T22:28:01.549526+0000","last_unstale":"2024-04-26T22:28:01.554873+0000","last_undegraded":"2024-04-26T22:28:01.554873+0000","last_fullsized":"2024-04-26T22:28:01.554873+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:28:36.732304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541224+0000","last_change":"2024-04-26T22:28:01.541224+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541224+0000","last_undegraded":"2024-04-26T22:28:01.541224+0000","last_fullsized":"2024-04-26T22:28:01.541224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558246+0000","last_change":"2024-04-26T22:28:01.555358+0000","last_active":"2024-04-26T22:28:01.558246+0000","last_peered":"2024-04-26T22:28:01.558246+0000","last_clean":"2024-04-26T22:28:01.558246+0000","last_became_active":"2024-04-26T22:28:01.555056+0000","last_became_peered":"2024-04-26T22:28:01.555056+0000","last_unstale":"2024-04-26T22:28:01.558246+0000","last_undegraded":"2024-04-26T22:28:01.558246+0000","last_fullsized":"2024-04-26T22:28:01.558246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547161+0000","last_change":"2024-04-26T22:28:01.547161+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547161+0000","last_undegraded":"2024-04-26T22:28:01.547161+0000","last_fullsized":"2024-04-26T22:28:01.547161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556304+0000","last_change":"2024-04-26T22:28:01.554918+0000","last_active":"2024-04-26T22:28:01.556304+0000","last_peered":"2024-04-26T22:28:01.556304+0000","last_clean":"2024-04-26T22:28:01.556304+0000","last_became_active":"2024-04-26T22:28:01.554637+0000","last_became_peered":"2024-04-26T22:28:01.554637+0000","last_unstale":"2024-04-26T22:28:01.556304+0000","last_undegraded":"2024-04-26T22:28:01.556304+0000","last_fullsized":"2024-04-26T22:28:01.556304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.550940+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:01.550940+0000","last_peered":"2024-04-26T22:28:01.550940+0000","last_clean":"2024-04-26T22:28:01.550940+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:01.550940+0000","last_undegraded":"2024-04-26T22:28:01.550940+0000","last_fullsized":"2024-04-26T22:28:01.550940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:06.568 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 Processor -- start 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 -- start start 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd406f790 0x7f8dd4074030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8dd415d780 0x7f8dd415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd4161710 con 0x7f8dd406f790 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd4161b80 con 0x7f8dd415d780 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.705+0000 7f8dd9571640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd40b4400 con 0x7f8dd4160d70 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd406f790 0x7f8dd4074030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f8dd406f790 0x7f8dd4074030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43350/0 (socket says 172.21.15.62:43350) 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/3587494106 learned_addr learned my addr 172.21.15.62:0/3587494106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd37fe640 1 --2- 172.21.15.62:0/3587494106 >> v2:172.21.15.174:3300/0 conn(0x7f8dd415d780 0x7f8dd415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd27fc640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/3587494106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 msgr2=0x7f8dd41611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/3587494106 >> v2:172.21.15.174:3300/0 conn(0x7f8dd415d780 msgr2=0x7f8dd415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:06.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/3587494106 >> v2:172.21.15.174:3300/0 conn(0x7f8dd415d780 0x7f8dd415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.708 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.706+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/3587494106 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd4074570 con 0x7f8dd406f790 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd4074030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4157df0 tx=0x7f8dbc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8481bc2cb3b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd1ffb640 1 -- 172.21.15.62:0/3587494106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc037070 con 0x7f8dd406f790 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd1ffb640 1 -- 172.21.15.62:0/3587494106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dbc032440 con 0x7f8dd406f790 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd1ffb640 1 -- 172.21.15.62:0/3587494106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc032810 con 0x7f8dd406f790 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd9571640 1 -- 172.21.15.62:0/3587494106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 msgr2=0x7f8dd4074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.707+0000 7f8dd9571640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd4074030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4157df0 tx=0x7f8dbc02a1e0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 -- 172.21.15.62:0/3587494106 shutdown_connections 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 --2- 172.21.15.62:0/3587494106 >> v2:172.21.15.174:3300/0 conn(0x7f8dd415d780 0x7f8dd415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 --2- 172.21.15.62:0/3587494106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd4074030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 -- 172.21.15.62:0/3587494106 >> 172.21.15.62:0/3587494106 conn(0x7f8dd40d0d50 msgr2=0x7f8dd40d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 -- 172.21.15.62:0/3587494106 shutdown_connections 2024-04-26T22:28:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.708+0000 7f8dd9571640 1 -- 172.21.15.62:0/3587494106 wait complete. 2024-04-26T22:28:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.709+0000 7f8dd9571640 1 Processor -- start 2024-04-26T22:28:06.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.709+0000 7f8dd9571640 1 -- start start 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd415d780 0x7f8dd415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd40732d0 con 0x7f8dd406f790 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd4072ff0 con 0x7f8dd415d780 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd40735b0 con 0x7f8dd4160d70 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43354/0 (socket says 172.21.15.62:43354) 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/2798774112 learned_addr learned my addr 172.21.15.62:0/2798774112 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 msgr2=0x7f8dd41f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd415d780 msgr2=0x7f8dd415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd27fc640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd415d780 0x7f8dd415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd415d780 0x7f8dd415c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 -- 172.21.15.62:0/2798774112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd41f35b0 con 0x7f8dd406f790 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd2ffd640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc000c00 tx=0x7f8dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc041020 con 0x7f8dd406f790 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dbc03d620 con 0x7f8dd406f790 2024-04-26T22:28:06.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.710+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd41f3890 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.711+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc037040 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.711+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd41f3e50 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.712+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd406a360 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.712+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dbc044020 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.712+0000 7f8dcb7fe640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.713+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f8dbc0ac190 con 0x7f8dd406f790 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.713+0000 7f8dd27fc640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:06.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.713+0000 7f8dd27fc640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b4630 tx=0x7f8dc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:06.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.717+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8dbc04b030 con 0x7f8dd406f790 2024-04-26T22:28:06.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.896+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8dd40ab2a0 con 0x7f8db406c8f0 2024-04-26T22:28:06.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.903+0000 7f8dcb7fe640 1 -- 172.21.15.62:0/2798774112 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7f8dd40ab2a0 con 0x7f8db406c8f0 2024-04-26T22:28:06.905 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:06.907 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 msgr2=0x7f8db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40b4630 tx=0x7f8dc4008040 comp rx=0 tx=0).stop 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 msgr2=0x7f8dd41f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc000c00 tx=0x7f8dbc0046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 shutdown_connections 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f8db406c8f0 0x7f8db406edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8dd4160d70 0x7f8dd41f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dd415d780 0x7f8dd415c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 --2- 172.21.15.62:0/2798774112 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8dd406f790 0x7f8dd41f2f60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:06.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 >> 172.21.15.62:0/2798774112 conn(0x7f8dd40d0d50 msgr2=0x7f8dd40b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:06.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 shutdown_connections 2024-04-26T22:28:06.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:06.906+0000 7f8dd9571640 1 -- 172.21.15.62:0/2798774112 wait complete. 2024-04-26T22:28:06.917 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-26T22:28:05.651527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2109795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.120179"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544043+0000","last_change":"2024-04-26T22:28:01.544043+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544043+0000","last_undegraded":"2024-04-26T22:28:01.544043+0000","last_fullsized":"2024-04-26T22:28:01.544043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.555306+0000","last_change":"2024-04-26T22:28:01.555306+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.555306+0000","last_undegraded":"2024-04-26T22:28:01.555306+0000","last_fullsized":"2024-04-26T22:28:01.555306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540215+0000","last_change":"2024-04-26T22:28:01.540215+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540215+0000","last_undegraded":"2024-04-26T22:28:01.540215+0000","last_fullsized":"2024-04-26T22:28:01.540215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550106+0000","last_change":"2024-04-26T22:28:01.550106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550106+0000","last_undegraded":"2024-04-26T22:28:01.550106+0000","last_fullsized":"2024-04-26T22:28:01.550106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544326+0000","last_change":"2024-04-26T22:28:01.544326+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544326+0000","last_undegraded":"2024-04-26T22:28:01.544326+0000","last_fullsized":"2024-04-26T22:28:01.544326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554580+0000","last_change":"2024-04-26T22:28:01.554580+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554580+0000","last_undegraded":"2024-04-26T22:28:01.554580+0000","last_fullsized":"2024-04-26T22:28:01.554580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547702+0000","last_change":"2024-04-26T22:28:01.547702+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547702+0000","last_undegraded":"2024-04-26T22:28:01.547702+0000","last_fullsized":"2024-04-26T22:28:01.547702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550612+0000","last_change":"2024-04-26T22:28:01.550612+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550612+0000","last_undegraded":"2024-04-26T22:28:01.550612+0000","last_fullsized":"2024-04-26T22:28:01.550612+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541137+0000","last_change":"2024-04-26T22:28:01.541137+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541137+0000","last_undegraded":"2024-04-26T22:28:01.541137+0000","last_fullsized":"2024-04-26T22:28:01.541137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550090+0000","last_change":"2024-04-26T22:28:01.550090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550090+0000","last_undegraded":"2024-04-26T22:28:01.550090+0000","last_fullsized":"2024-04-26T22:28:01.550090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546936+0000","last_change":"2024-04-26T22:28:01.546936+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546936+0000","last_undegraded":"2024-04-26T22:28:01.546936+0000","last_fullsized":"2024-04-26T22:28:01.546936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543309+0000","last_change":"2024-04-26T22:28:01.543309+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543309+0000","last_undegraded":"2024-04-26T22:28:01.543309+0000","last_fullsized":"2024-04-26T22:28:01.543309+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543892+0000","last_change":"2024-04-26T22:28:01.543892+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543892+0000","last_undegraded":"2024-04-26T22:28:01.543892+0000","last_fullsized":"2024-04-26T22:28:01.543892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554052+0000","last_change":"2024-04-26T22:28:01.554052+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554052+0000","last_undegraded":"2024-04-26T22:28:01.554052+0000","last_fullsized":"2024-04-26T22:28:01.554052+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540104+0000","last_change":"2024-04-26T22:28:01.540104+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540104+0000","last_undegraded":"2024-04-26T22:28:01.540104+0000","last_fullsized":"2024-04-26T22:28:01.540104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546355+0000","last_change":"2024-04-26T22:28:01.546355+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546355+0000","last_undegraded":"2024-04-26T22:28:01.546355+0000","last_fullsized":"2024-04-26T22:28:01.546355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546942+0000","last_change":"2024-04-26T22:28:01.546942+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546942+0000","last_undegraded":"2024-04-26T22:28:01.546942+0000","last_fullsized":"2024-04-26T22:28:01.546942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552774+0000","last_change":"2024-04-26T22:28:01.552774+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552774+0000","last_undegraded":"2024-04-26T22:28:01.552774+0000","last_fullsized":"2024-04-26T22:28:01.552774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552113+0000","last_change":"2024-04-26T22:28:01.552113+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552113+0000","last_undegraded":"2024-04-26T22:28:01.552113+0000","last_fullsized":"2024-04-26T22:28:01.552113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542939+0000","last_change":"2024-04-26T22:28:01.542939+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542939+0000","last_undegraded":"2024-04-26T22:28:01.542939+0000","last_fullsized":"2024-04-26T22:28:01.542939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543242+0000","last_change":"2024-04-26T22:28:01.543242+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543242+0000","last_undegraded":"2024-04-26T22:28:01.543242+0000","last_fullsized":"2024-04-26T22:28:01.543242+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540859+0000","last_change":"2024-04-26T22:28:01.540859+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540859+0000","last_undegraded":"2024-04-26T22:28:01.540859+0000","last_fullsized":"2024-04-26T22:28:01.540859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546687+0000","last_change":"2024-04-26T22:28:01.546687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546687+0000","last_undegraded":"2024-04-26T22:28:01.546687+0000","last_fullsized":"2024-04-26T22:28:01.546687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540745+0000","last_change":"2024-04-26T22:28:01.540745+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540745+0000","last_undegraded":"2024-04-26T22:28:01.540745+0000","last_fullsized":"2024-04-26T22:28:01.540745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550111+0000","last_change":"2024-04-26T22:28:01.550111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550111+0000","last_undegraded":"2024-04-26T22:28:01.550111+0000","last_fullsized":"2024-04-26T22:28:01.550111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551624+0000","last_change":"2024-04-26T22:28:01.551624+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551624+0000","last_undegraded":"2024-04-26T22:28:01.551624+0000","last_fullsized":"2024-04-26T22:28:01.551624+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550886+0000","last_change":"2024-04-26T22:28:01.550886+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550886+0000","last_undegraded":"2024-04-26T22:28:01.550886+0000","last_fullsized":"2024-04-26T22:28:01.550886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540303+0000","last_change":"2024-04-26T22:28:01.540303+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540303+0000","last_undegraded":"2024-04-26T22:28:01.540303+0000","last_fullsized":"2024-04-26T22:28:01.540303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541775+0000","last_change":"2024-04-26T22:28:01.541775+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541775+0000","last_undegraded":"2024-04-26T22:28:01.541775+0000","last_fullsized":"2024-04-26T22:28:01.541775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546895+0000","last_change":"2024-04-26T22:28:01.546895+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546895+0000","last_undegraded":"2024-04-26T22:28:01.546895+0000","last_fullsized":"2024-04-26T22:28:01.546895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543109+0000","last_change":"2024-04-26T22:28:01.543109+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543109+0000","last_undegraded":"2024-04-26T22:28:01.543109+0000","last_fullsized":"2024-04-26T22:28:01.543109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550216+0000","last_change":"2024-04-26T22:28:01.550216+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550216+0000","last_undegraded":"2024-04-26T22:28:01.550216+0000","last_fullsized":"2024-04-26T22:28:01.550216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547102+0000","last_change":"2024-04-26T22:28:01.547102+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547102+0000","last_undegraded":"2024-04-26T22:28:01.547102+0000","last_fullsized":"2024-04-26T22:28:01.547102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554323+0000","last_change":"2024-04-26T22:28:01.551148+0000","last_active":"2024-04-26T22:28:01.554323+0000","last_peered":"2024-04-26T22:28:01.554323+0000","last_clean":"2024-04-26T22:28:01.554323+0000","last_became_active":"2024-04-26T22:28:01.550854+0000","last_became_peered":"2024-04-26T22:28:01.550854+0000","last_unstale":"2024-04-26T22:28:01.554323+0000","last_undegraded":"2024-04-26T22:28:01.554323+0000","last_fullsized":"2024-04-26T22:28:01.554323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543631+0000","last_change":"2024-04-26T22:28:01.543631+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543631+0000","last_undegraded":"2024-04-26T22:28:01.543631+0000","last_fullsized":"2024-04-26T22:28:01.543631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556848+0000","last_change":"2024-04-26T22:28:01.553286+0000","last_active":"2024-04-26T22:28:01.556848+0000","last_peered":"2024-04-26T22:28:01.556848+0000","last_clean":"2024-04-26T22:28:01.556848+0000","last_became_active":"2024-04-26T22:28:01.552926+0000","last_became_peered":"2024-04-26T22:28:01.552926+0000","last_unstale":"2024-04-26T22:28:01.556848+0000","last_undegraded":"2024-04-26T22:28:01.556848+0000","last_fullsized":"2024-04-26T22:28:01.556848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541804+0000","last_change":"2024-04-26T22:28:01.541804+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541804+0000","last_undegraded":"2024-04-26T22:28:01.541804+0000","last_fullsized":"2024-04-26T22:28:01.541804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566207+0000","last_change":"2024-04-26T22:28:01.559844+0000","last_active":"2024-04-26T22:28:01.566207+0000","last_peered":"2024-04-26T22:28:01.566207+0000","last_clean":"2024-04-26T22:28:01.566207+0000","last_became_active":"2024-04-26T22:28:01.559546+0000","last_became_peered":"2024-04-26T22:28:01.559546+0000","last_unstale":"2024-04-26T22:28:01.566207+0000","last_undegraded":"2024-04-26T22:28:01.566207+0000","last_fullsized":"2024-04-26T22:28:01.566207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:59.038151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550464+0000","last_change":"2024-04-26T22:28:01.550464+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550464+0000","last_undegraded":"2024-04-26T22:28:01.550464+0000","last_fullsized":"2024-04-26T22:28:01.550464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554674+0000","last_change":"2024-04-26T22:28:01.552326+0000","last_active":"2024-04-26T22:28:01.554674+0000","last_peered":"2024-04-26T22:28:01.554674+0000","last_clean":"2024-04-26T22:28:01.554674+0000","last_became_active":"2024-04-26T22:28:01.551423+0000","last_became_peered":"2024-04-26T22:28:01.551423+0000","last_unstale":"2024-04-26T22:28:01.554674+0000","last_undegraded":"2024-04-26T22:28:01.554674+0000","last_fullsized":"2024-04-26T22:28:01.554674+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:17.520923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546338+0000","last_change":"2024-04-26T22:28:01.546338+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546338+0000","last_undegraded":"2024-04-26T22:28:01.546338+0000","last_fullsized":"2024-04-26T22:28:01.546338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554857+0000","last_change":"2024-04-26T22:28:01.549390+0000","last_active":"2024-04-26T22:28:01.554857+0000","last_peered":"2024-04-26T22:28:01.554857+0000","last_clean":"2024-04-26T22:28:01.554857+0000","last_became_active":"2024-04-26T22:28:01.548063+0000","last_became_peered":"2024-04-26T22:28:01.548063+0000","last_unstale":"2024-04-26T22:28:01.554857+0000","last_undegraded":"2024-04-26T22:28:01.554857+0000","last_fullsized":"2024-04-26T22:28:01.554857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.079402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540460+0000","last_change":"2024-04-26T22:28:01.540460+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540460+0000","last_undegraded":"2024-04-26T22:28:01.540460+0000","last_fullsized":"2024-04-26T22:28:01.540460+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553031+0000","last_change":"2024-04-26T22:28:01.548404+0000","last_active":"2024-04-26T22:28:01.553031+0000","last_peered":"2024-04-26T22:28:01.553031+0000","last_clean":"2024-04-26T22:28:01.553031+0000","last_became_active":"2024-04-26T22:28:01.548088+0000","last_became_peered":"2024-04-26T22:28:01.548088+0000","last_unstale":"2024-04-26T22:28:01.553031+0000","last_undegraded":"2024-04-26T22:28:01.553031+0000","last_fullsized":"2024-04-26T22:28:01.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:38.650628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542687+0000","last_change":"2024-04-26T22:28:01.542687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542687+0000","last_undegraded":"2024-04-26T22:28:01.542687+0000","last_fullsized":"2024-04-26T22:28:01.542687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563282+0000","last_change":"2024-04-26T22:28:01.556939+0000","last_active":"2024-04-26T22:28:01.563282+0000","last_peered":"2024-04-26T22:28:01.563282+0000","last_clean":"2024-04-26T22:28:01.563282+0000","last_became_active":"2024-04-26T22:28:01.556663+0000","last_became_peered":"2024-04-26T22:28:01.556663+0000","last_unstale":"2024-04-26T22:28:01.563282+0000","last_undegraded":"2024-04-26T22:28:01.563282+0000","last_fullsized":"2024-04-26T22:28:01.563282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549555+0000","last_change":"2024-04-26T22:28:01.549555+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549555+0000","last_undegraded":"2024-04-26T22:28:01.549555+0000","last_fullsized":"2024-04-26T22:28:01.549555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554932+0000","last_change":"2024-04-26T22:28:01.552353+0000","last_active":"2024-04-26T22:28:01.554932+0000","last_peered":"2024-04-26T22:28:01.554932+0000","last_clean":"2024-04-26T22:28:01.554932+0000","last_became_active":"2024-04-26T22:28:01.549925+0000","last_became_peered":"2024-04-26T22:28:01.549925+0000","last_unstale":"2024-04-26T22:28:01.554932+0000","last_undegraded":"2024-04-26T22:28:01.554932+0000","last_fullsized":"2024-04-26T22:28:01.554932+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:42.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543786+0000","last_change":"2024-04-26T22:28:01.543786+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543786+0000","last_undegraded":"2024-04-26T22:28:01.543786+0000","last_fullsized":"2024-04-26T22:28:01.543786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558468+0000","last_change":"2024-04-26T22:28:01.555381+0000","last_active":"2024-04-26T22:28:01.558468+0000","last_peered":"2024-04-26T22:28:01.558468+0000","last_clean":"2024-04-26T22:28:01.558468+0000","last_became_active":"2024-04-26T22:28:01.555132+0000","last_became_peered":"2024-04-26T22:28:01.555132+0000","last_unstale":"2024-04-26T22:28:01.558468+0000","last_undegraded":"2024-04-26T22:28:01.558468+0000","last_fullsized":"2024-04-26T22:28:01.558468+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.553709+0000","last_change":"2024-04-26T22:28:01.553709+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.553709+0000","last_undegraded":"2024-04-26T22:28:01.553709+0000","last_fullsized":"2024-04-26T22:28:01.553709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566486+0000","last_change":"2024-04-26T22:28:01.563856+0000","last_active":"2024-04-26T22:28:01.566486+0000","last_peered":"2024-04-26T22:28:01.566486+0000","last_clean":"2024-04-26T22:28:01.566486+0000","last_became_active":"2024-04-26T22:28:01.563514+0000","last_became_peered":"2024-04-26T22:28:01.563514+0000","last_unstale":"2024-04-26T22:28:01.566486+0000","last_undegraded":"2024-04-26T22:28:01.566486+0000","last_fullsized":"2024-04-26T22:28:01.566486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551090+0000","last_change":"2024-04-26T22:28:01.551090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551090+0000","last_undegraded":"2024-04-26T22:28:01.551090+0000","last_fullsized":"2024-04-26T22:28:01.551090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555437+0000","last_change":"2024-04-26T22:28:01.553958+0000","last_active":"2024-04-26T22:28:01.555437+0000","last_peered":"2024-04-26T22:28:01.555437+0000","last_clean":"2024-04-26T22:28:01.555437+0000","last_became_active":"2024-04-26T22:28:01.553552+0000","last_became_peered":"2024-04-26T22:28:01.553552+0000","last_unstale":"2024-04-26T22:28:01.555437+0000","last_undegraded":"2024-04-26T22:28:01.555437+0000","last_fullsized":"2024-04-26T22:28:01.555437+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540973+0000","last_change":"2024-04-26T22:28:01.540973+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540973+0000","last_undegraded":"2024-04-26T22:28:01.540973+0000","last_fullsized":"2024-04-26T22:28:01.540973+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566167+0000","last_change":"2024-04-26T22:28:01.559144+0000","last_active":"2024-04-26T22:28:01.566167+0000","last_peered":"2024-04-26T22:28:01.566167+0000","last_clean":"2024-04-26T22:28:01.566167+0000","last_became_active":"2024-04-26T22:28:01.558496+0000","last_became_peered":"2024-04-26T22:28:01.558496+0000","last_unstale":"2024-04-26T22:28:01.566167+0000","last_undegraded":"2024-04-26T22:28:01.566167+0000","last_fullsized":"2024-04-26T22:28:01.566167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551099+0000","last_change":"2024-04-26T22:28:01.551099+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551099+0000","last_undegraded":"2024-04-26T22:28:01.551099+0000","last_fullsized":"2024-04-26T22:28:01.551099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554892+0000","last_change":"2024-04-26T22:28:01.550978+0000","last_active":"2024-04-26T22:28:01.554892+0000","last_peered":"2024-04-26T22:28:01.554892+0000","last_clean":"2024-04-26T22:28:01.554892+0000","last_became_active":"2024-04-26T22:28:01.550154+0000","last_became_peered":"2024-04-26T22:28:01.550154+0000","last_unstale":"2024-04-26T22:28:01.554892+0000","last_undegraded":"2024-04-26T22:28:01.554892+0000","last_fullsized":"2024-04-26T22:28:01.554892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:59.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550436+0000","last_change":"2024-04-26T22:28:01.550436+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550436+0000","last_undegraded":"2024-04-26T22:28:01.550436+0000","last_fullsized":"2024-04-26T22:28:01.550436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559137+0000","last_change":"2024-04-26T22:28:01.551899+0000","last_active":"2024-04-26T22:28:01.559137+0000","last_peered":"2024-04-26T22:28:01.559137+0000","last_clean":"2024-04-26T22:28:01.559137+0000","last_became_active":"2024-04-26T22:28:01.551587+0000","last_became_peered":"2024-04-26T22:28:01.551587+0000","last_unstale":"2024-04-26T22:28:01.559137+0000","last_undegraded":"2024-04-26T22:28:01.559137+0000","last_fullsized":"2024-04-26T22:28:01.559137+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:09.972664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549849+0000","last_change":"2024-04-26T22:28:01.549849+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549849+0000","last_undegraded":"2024-04-26T22:28:01.549849+0000","last_fullsized":"2024-04-26T22:28:01.549849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558181+0000","last_change":"2024-04-26T22:28:01.555910+0000","last_active":"2024-04-26T22:28:01.558181+0000","last_peered":"2024-04-26T22:28:01.558181+0000","last_clean":"2024-04-26T22:28:01.558181+0000","last_became_active":"2024-04-26T22:28:01.555596+0000","last_became_peered":"2024-04-26T22:28:01.555596+0000","last_unstale":"2024-04-26T22:28:01.558181+0000","last_undegraded":"2024-04-26T22:28:01.558181+0000","last_fullsized":"2024-04-26T22:28:01.558181+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542675+0000","last_change":"2024-04-26T22:28:01.542675+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542675+0000","last_undegraded":"2024-04-26T22:28:01.542675+0000","last_fullsized":"2024-04-26T22:28:01.542675+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566270+0000","last_change":"2024-04-26T22:28:01.559695+0000","last_active":"2024-04-26T22:28:01.566270+0000","last_peered":"2024-04-26T22:28:01.566270+0000","last_clean":"2024-04-26T22:28:01.566270+0000","last_became_active":"2024-04-26T22:28:01.559448+0000","last_became_peered":"2024-04-26T22:28:01.559448+0000","last_unstale":"2024-04-26T22:28:01.566270+0000","last_undegraded":"2024-04-26T22:28:01.566270+0000","last_fullsized":"2024-04-26T22:28:01.566270+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542254+0000","last_change":"2024-04-26T22:28:01.542254+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542254+0000","last_undegraded":"2024-04-26T22:28:01.542254+0000","last_fullsized":"2024-04-26T22:28:01.542254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566076+0000","last_change":"2024-04-26T22:28:01.560134+0000","last_active":"2024-04-26T22:28:01.566076+0000","last_peered":"2024-04-26T22:28:01.566076+0000","last_clean":"2024-04-26T22:28:01.566076+0000","last_became_active":"2024-04-26T22:28:01.559832+0000","last_became_peered":"2024-04-26T22:28:01.559832+0000","last_unstale":"2024-04-26T22:28:01.566076+0000","last_undegraded":"2024-04-26T22:28:01.566076+0000","last_fullsized":"2024-04-26T22:28:01.566076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542106+0000","last_change":"2024-04-26T22:28:01.542106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542106+0000","last_undegraded":"2024-04-26T22:28:01.542106+0000","last_fullsized":"2024-04-26T22:28:01.542106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554944+0000","last_change":"2024-04-26T22:28:01.550770+0000","last_active":"2024-04-26T22:28:01.554944+0000","last_peered":"2024-04-26T22:28:01.554944+0000","last_clean":"2024-04-26T22:28:01.554944+0000","last_became_active":"2024-04-26T22:28:01.550490+0000","last_became_peered":"2024-04-26T22:28:01.550490+0000","last_unstale":"2024-04-26T22:28:01.554944+0000","last_undegraded":"2024-04-26T22:28:01.554944+0000","last_fullsized":"2024-04-26T22:28:01.554944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:29.207017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549567+0000","last_change":"2024-04-26T22:28:01.549567+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549567+0000","last_undegraded":"2024-04-26T22:28:01.549567+0000","last_fullsized":"2024-04-26T22:28:01.549567+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553033+0000","last_change":"2024-04-26T22:28:01.548123+0000","last_active":"2024-04-26T22:28:01.553033+0000","last_peered":"2024-04-26T22:28:01.553033+0000","last_clean":"2024-04-26T22:28:01.553033+0000","last_became_active":"2024-04-26T22:28:01.547847+0000","last_became_peered":"2024-04-26T22:28:01.547847+0000","last_unstale":"2024-04-26T22:28:01.553033+0000","last_undegraded":"2024-04-26T22:28:01.553033+0000","last_fullsized":"2024-04-26T22:28:01.553033+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:50.283072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543141+0000","last_change":"2024-04-26T22:28:01.543141+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543141+0000","last_undegraded":"2024-04-26T22:28:01.543141+0000","last_fullsized":"2024-04-26T22:28:01.543141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554806+0000","last_change":"2024-04-26T22:28:01.548757+0000","last_active":"2024-04-26T22:28:01.554806+0000","last_peered":"2024-04-26T22:28:01.554806+0000","last_clean":"2024-04-26T22:28:01.554806+0000","last_became_active":"2024-04-26T22:28:01.548498+0000","last_became_peered":"2024-04-26T22:28:01.548498+0000","last_unstale":"2024-04-26T22:28:01.554806+0000","last_undegraded":"2024-04-26T22:28:01.554806+0000","last_fullsized":"2024-04-26T22:28:01.554806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.207881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543163+0000","last_change":"2024-04-26T22:28:01.543163+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543163+0000","last_undegraded":"2024-04-26T22:28:01.543163+0000","last_fullsized":"2024-04-26T22:28:01.543163+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.560904+0000","last_change":"2024-04-26T22:28:01.559890+0000","last_active":"2024-04-26T22:28:01.560904+0000","last_peered":"2024-04-26T22:28:01.560904+0000","last_clean":"2024-04-26T22:28:01.560904+0000","last_became_active":"2024-04-26T22:28:01.559564+0000","last_became_peered":"2024-04-26T22:28:01.559564+0000","last_unstale":"2024-04-26T22:28:01.560904+0000","last_undegraded":"2024-04-26T22:28:01.560904+0000","last_fullsized":"2024-04-26T22:28:01.560904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543175+0000","last_change":"2024-04-26T22:28:01.543175+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543175+0000","last_undegraded":"2024-04-26T22:28:01.543175+0000","last_fullsized":"2024-04-26T22:28:01.543175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559529+0000","last_change":"2024-04-26T22:28:01.558972+0000","last_active":"2024-04-26T22:28:01.559529+0000","last_peered":"2024-04-26T22:28:01.559529+0000","last_clean":"2024-04-26T22:28:01.559529+0000","last_became_active":"2024-04-26T22:28:01.558693+0000","last_became_peered":"2024-04-26T22:28:01.558693+0000","last_unstale":"2024-04-26T22:28:01.559529+0000","last_undegraded":"2024-04-26T22:28:01.559529+0000","last_fullsized":"2024-04-26T22:28:01.559529+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542997+0000","last_change":"2024-04-26T22:28:01.542997+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542997+0000","last_undegraded":"2024-04-26T22:28:01.542997+0000","last_fullsized":"2024-04-26T22:28:01.542997+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555059+0000","last_change":"2024-04-26T22:28:01.551281+0000","last_active":"2024-04-26T22:28:01.555059+0000","last_peered":"2024-04-26T22:28:01.555059+0000","last_clean":"2024-04-26T22:28:01.555059+0000","last_became_active":"2024-04-26T22:28:01.550195+0000","last_became_peered":"2024-04-26T22:28:01.550195+0000","last_unstale":"2024-04-26T22:28:01.555059+0000","last_undegraded":"2024-04-26T22:28:01.555059+0000","last_fullsized":"2024-04-26T22:28:01.555059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:58.163201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.548111+0000","last_change":"2024-04-26T22:28:01.548111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.548111+0000","last_undegraded":"2024-04-26T22:28:01.548111+0000","last_fullsized":"2024-04-26T22:28:01.548111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555367+0000","last_change":"2024-04-26T22:28:01.553831+0000","last_active":"2024-04-26T22:28:01.555367+0000","last_peered":"2024-04-26T22:28:01.555367+0000","last_clean":"2024-04-26T22:28:01.555367+0000","last_became_active":"2024-04-26T22:28:01.553563+0000","last_became_peered":"2024-04-26T22:28:01.553563+0000","last_unstale":"2024-04-26T22:28:01.555367+0000","last_undegraded":"2024-04-26T22:28:01.555367+0000","last_fullsized":"2024-04-26T22:28:01.555367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543247+0000","last_change":"2024-04-26T22:28:01.543247+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543247+0000","last_undegraded":"2024-04-26T22:28:01.543247+0000","last_fullsized":"2024-04-26T22:28:01.543247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565921+0000","last_change":"2024-04-26T22:28:01.558893+0000","last_active":"2024-04-26T22:28:01.565921+0000","last_peered":"2024-04-26T22:28:01.565921+0000","last_clean":"2024-04-26T22:28:01.565921+0000","last_became_active":"2024-04-26T22:28:01.558541+0000","last_became_peered":"2024-04-26T22:28:01.558541+0000","last_unstale":"2024-04-26T22:28:01.565921+0000","last_undegraded":"2024-04-26T22:28:01.565921+0000","last_fullsized":"2024-04-26T22:28:01.565921+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540152+0000","last_change":"2024-04-26T22:28:01.540152+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540152+0000","last_undegraded":"2024-04-26T22:28:01.540152+0000","last_fullsized":"2024-04-26T22:28:01.540152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563366+0000","last_change":"2024-04-26T22:28:01.560436+0000","last_active":"2024-04-26T22:28:01.563366+0000","last_peered":"2024-04-26T22:28:01.563366+0000","last_clean":"2024-04-26T22:28:01.563366+0000","last_became_active":"2024-04-26T22:28:01.560153+0000","last_became_peered":"2024-04-26T22:28:01.560153+0000","last_unstale":"2024-04-26T22:28:01.563366+0000","last_undegraded":"2024-04-26T22:28:01.563366+0000","last_fullsized":"2024-04-26T22:28:01.563366+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546676+0000","last_change":"2024-04-26T22:28:01.546676+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546676+0000","last_undegraded":"2024-04-26T22:28:01.546676+0000","last_fullsized":"2024-04-26T22:28:01.546676+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565905+0000","last_change":"2024-04-26T22:28:01.558609+0000","last_active":"2024-04-26T22:28:01.565905+0000","last_peered":"2024-04-26T22:28:01.565905+0000","last_clean":"2024-04-26T22:28:01.565905+0000","last_became_active":"2024-04-26T22:28:01.558332+0000","last_became_peered":"2024-04-26T22:28:01.558332+0000","last_unstale":"2024-04-26T22:28:01.565905+0000","last_undegraded":"2024-04-26T22:28:01.565905+0000","last_fullsized":"2024-04-26T22:28:01.565905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543092+0000","last_change":"2024-04-26T22:28:01.543092+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543092+0000","last_undegraded":"2024-04-26T22:28:01.543092+0000","last_fullsized":"2024-04-26T22:28:01.543092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556844+0000","last_change":"2024-04-26T22:28:01.551370+0000","last_active":"2024-04-26T22:28:01.556844+0000","last_peered":"2024-04-26T22:28:01.556844+0000","last_clean":"2024-04-26T22:28:01.556844+0000","last_became_active":"2024-04-26T22:28:01.550977+0000","last_became_peered":"2024-04-26T22:28:01.550977+0000","last_unstale":"2024-04-26T22:28:01.556844+0000","last_undegraded":"2024-04-26T22:28:01.556844+0000","last_fullsized":"2024-04-26T22:28:01.556844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546076+0000","last_change":"2024-04-26T22:28:01.546076+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546076+0000","last_undegraded":"2024-04-26T22:28:01.546076+0000","last_fullsized":"2024-04-26T22:28:01.546076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559355+0000","last_change":"2024-04-26T22:28:01.555714+0000","last_active":"2024-04-26T22:28:01.559355+0000","last_peered":"2024-04-26T22:28:01.559355+0000","last_clean":"2024-04-26T22:28:01.559355+0000","last_became_active":"2024-04-26T22:28:01.555437+0000","last_became_peered":"2024-04-26T22:28:01.555437+0000","last_unstale":"2024-04-26T22:28:01.559355+0000","last_undegraded":"2024-04-26T22:28:01.559355+0000","last_fullsized":"2024-04-26T22:28:01.559355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549204+0000","last_change":"2024-04-26T22:28:01.549204+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549204+0000","last_undegraded":"2024-04-26T22:28:01.549204+0000","last_fullsized":"2024-04-26T22:28:01.549204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554873+0000","last_change":"2024-04-26T22:28:01.549998+0000","last_active":"2024-04-26T22:28:01.554873+0000","last_peered":"2024-04-26T22:28:01.554873+0000","last_clean":"2024-04-26T22:28:01.554873+0000","last_became_active":"2024-04-26T22:28:01.549526+0000","last_became_peered":"2024-04-26T22:28:01.549526+0000","last_unstale":"2024-04-26T22:28:01.554873+0000","last_undegraded":"2024-04-26T22:28:01.554873+0000","last_fullsized":"2024-04-26T22:28:01.554873+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:28:36.732304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541224+0000","last_change":"2024-04-26T22:28:01.541224+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541224+0000","last_undegraded":"2024-04-26T22:28:01.541224+0000","last_fullsized":"2024-04-26T22:28:01.541224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558246+0000","last_change":"2024-04-26T22:28:01.555358+0000","last_active":"2024-04-26T22:28:01.558246+0000","last_peered":"2024-04-26T22:28:01.558246+0000","last_clean":"2024-04-26T22:28:01.558246+0000","last_became_active":"2024-04-26T22:28:01.555056+0000","last_became_peered":"2024-04-26T22:28:01.555056+0000","last_unstale":"2024-04-26T22:28:01.558246+0000","last_undegraded":"2024-04-26T22:28:01.558246+0000","last_fullsized":"2024-04-26T22:28:01.558246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547161+0000","last_change":"2024-04-26T22:28:01.547161+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547161+0000","last_undegraded":"2024-04-26T22:28:01.547161+0000","last_fullsized":"2024-04-26T22:28:01.547161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556304+0000","last_change":"2024-04-26T22:28:01.554918+0000","last_active":"2024-04-26T22:28:01.556304+0000","last_peered":"2024-04-26T22:28:01.556304+0000","last_clean":"2024-04-26T22:28:01.556304+0000","last_became_active":"2024-04-26T22:28:01.554637+0000","last_became_peered":"2024-04-26T22:28:01.554637+0000","last_unstale":"2024-04-26T22:28:01.556304+0000","last_undegraded":"2024-04-26T22:28:01.556304+0000","last_fullsized":"2024-04-26T22:28:01.556304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.550940+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:01.550940+0000","last_peered":"2024-04-26T22:28:01.550940+0000","last_clean":"2024-04-26T22:28:01.550940+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:01.550940+0000","last_undegraded":"2024-04-26T22:28:01.550940+0000","last_fullsized":"2024-04-26T22:28:01.550940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:06.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:07.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.057+0000 7fbb401f7640 1 Processor -- start 2024-04-26T22:28:07.058 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.058+0000 7fbb401f7640 1 -- start start 2024-04-26T22:28:07.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.058+0000 7fbb401f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.058+0000 7fbb401f7640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb380d2670 con 0x7fbb380d3820 2024-04-26T22:28:07.059 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.058+0000 7fbb401f7640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb380d0210 con 0x7fbb380d3c80 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb401f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380d04f0 con 0x7fbb38162b40 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3d792640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbb380d3820 0x7fbb380d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42348/0 (socket says 172.21.15.62:42348) 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3d792640 1 -- 172.21.15.62:0/3281950014 learned_addr learned my addr 172.21.15.62:0/3281950014 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3df93640 1 --2- 172.21.15.62:0/3281950014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3943280319 0 0) 0x7fbb380d2670 con 0x7fbb380d3820 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb24003520 con 0x7fbb380d3820 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1324167384 0 0) 0x7fbb380d0210 con 0x7fbb380d3c80 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb380d2670 con 0x7fbb380d3c80 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405147887 0 0) 0x7fbb24003520 con 0x7fbb380d3820 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3df93640 1 -- 172.21.15.62:0/3281950014 >> v1:172.21.15.174:6789/0 conn(0x7fbb380d3c80 legacy=0x7fbb380d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3df93640 1 -- 172.21.15.62:0/3281950014 >> v1:172.21.15.62:6789/0 conn(0x7fbb380d3820 legacy=0x7fbb380d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.060 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.059+0000 7fbb3df93640 1 -- 172.21.15.62:0/3281950014 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380d07d0 con 0x7fbb38162b40 2024-04-26T22:28:07.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.060+0000 7fbb3df93640 1 --2- 172.21.15.62:0/3281950014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380d5bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097d0 tx=0x7fbb3002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f41db8abc70139 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.060+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30037070 con 0x7fbb38162b40 2024-04-26T22:28:07.061 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.060+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb3002acc0 con 0x7fbb38162b40 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.060+0000 7fbb3cf91640 1 -- 172.21.15.62:0/3281950014 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28001670 con 0x7fbb38162b40 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 -- 172.21.15.62:0/3281950014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 msgr2=0x7fbb380d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3281950014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380d5bd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb300097d0 tx=0x7fbb3002a230 comp rx=0 tx=0).stop 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 -- 172.21.15.62:0/3281950014 shutdown_connections 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3281950014 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380d5bd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 -- 172.21.15.62:0/3281950014 >> 172.21.15.62:0/3281950014 conn(0x7fbb3809e550 msgr2=0x7fbb380d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 -- 172.21.15.62:0/3281950014 shutdown_connections 2024-04-26T22:28:07.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.061+0000 7fbb401f7640 1 -- 172.21.15.62:0/3281950014 wait complete. 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.062+0000 7fbb401f7640 1 Processor -- start 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.062+0000 7fbb401f7640 1 -- start start 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 0x7fbb380ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380d4110 con 0x7fbb380d3820 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb380d0d90 con 0x7fbb380d3c80 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb401f7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbb380b5bb0 con 0x7fbb38162b40 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43356/0 (socket says 172.21.15.62:43356) 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 -- 172.21.15.62:0/3054546336 learned_addr learned my addr 172.21.15.62:0/3054546336 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3d792640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 0x7fbb380ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 -- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 msgr2=0x7fbb380cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3e794640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 -- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 msgr2=0x7fbb380ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 0x7fbb380ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 -- 172.21.15.62:0/3054546336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb380b9700 con 0x7fbb380d3820 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3e794640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3d792640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 0x7fbb380ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:07.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.063+0000 7fbb3df93640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbb300098f0 tx=0x7fbb300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30037050 con 0x7fbb380d3820 2024-04-26T22:28:07.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb30043070 con 0x7fbb380d3820 2024-04-26T22:28:07.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30006e10 con 0x7fbb380d3820 2024-04-26T22:28:07.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb380b99e0 con 0x7fbb380d3820 2024-04-26T22:28:07.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb380b9fa0 con 0x7fbb380d3820 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.064+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3806a360 con 0x7fbb380d3820 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.065+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb3003e690 con 0x7fbb380d3820 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.065+0000 7fbb2e7fc640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 0x7fbb1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.065+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbb300ac3f0 con 0x7fbb380d3820 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.066+0000 7fbb3d792640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 0x7fbb1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.066+0000 7fbb3d792640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 0x7fbb1c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb280017e0 tx=0x7fbb28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.070+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb3007a8b0 con 0x7fbb380d3820 2024-04-26T22:28:07.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.254+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb38162360 con 0x7fbb1c06c8f0 2024-04-26T22:28:07.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.261+0000 7fbb2e7fc640 1 -- 172.21.15.62:0/3054546336 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246632 (secure 0 0 0) 0x7fbb38162360 con 0x7fbb1c06c8f0 2024-04-26T22:28:07.263 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:07.264 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:07.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 msgr2=0x7fbb1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 0x7fbb1c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbb280017e0 tx=0x7fbb28009040 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 msgr2=0x7fbb380b8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbb300098f0 tx=0x7fbb300046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 shutdown_connections 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbb1c06c8f0 0x7fbb1c06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbb38162b40 0x7fbb380cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb380d3c80 0x7fbb380ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 --2- 172.21.15.62:0/3054546336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb380d3820 0x7fbb380b8830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 >> 172.21.15.62:0/3054546336 conn(0x7fbb3809e550 msgr2=0x7fbb380d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 shutdown_connections 2024-04-26T22:28:07.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.264+0000 7fbb401f7640 1 -- 172.21.15.62:0/3054546336 wait complete. 2024-04-26T22:28:07.275 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-04-26T22:28:05.651527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219768,"kb_used_data":3604,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754952,"statfs":{"total":773094113280,"available":772869070848,"internally_reserved":0,"allocated":3690496,"data_stored":2109795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.120179"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544043+0000","last_change":"2024-04-26T22:28:01.544043+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544043+0000","last_undegraded":"2024-04-26T22:28:01.544043+0000","last_fullsized":"2024-04-26T22:28:01.544043+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.555306+0000","last_change":"2024-04-26T22:28:01.555306+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.555306+0000","last_undegraded":"2024-04-26T22:28:01.555306+0000","last_fullsized":"2024-04-26T22:28:01.555306+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540215+0000","last_change":"2024-04-26T22:28:01.540215+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540215+0000","last_undegraded":"2024-04-26T22:28:01.540215+0000","last_fullsized":"2024-04-26T22:28:01.540215+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550106+0000","last_change":"2024-04-26T22:28:01.550106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550106+0000","last_undegraded":"2024-04-26T22:28:01.550106+0000","last_fullsized":"2024-04-26T22:28:01.550106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.544326+0000","last_change":"2024-04-26T22:28:01.544326+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.544326+0000","last_undegraded":"2024-04-26T22:28:01.544326+0000","last_fullsized":"2024-04-26T22:28:01.544326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554580+0000","last_change":"2024-04-26T22:28:01.554580+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554580+0000","last_undegraded":"2024-04-26T22:28:01.554580+0000","last_fullsized":"2024-04-26T22:28:01.554580+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547702+0000","last_change":"2024-04-26T22:28:01.547702+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547702+0000","last_undegraded":"2024-04-26T22:28:01.547702+0000","last_fullsized":"2024-04-26T22:28:01.547702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550612+0000","last_change":"2024-04-26T22:28:01.550612+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550612+0000","last_undegraded":"2024-04-26T22:28:01.550612+0000","last_fullsized":"2024-04-26T22:28:01.550612+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541137+0000","last_change":"2024-04-26T22:28:01.541137+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541137+0000","last_undegraded":"2024-04-26T22:28:01.541137+0000","last_fullsized":"2024-04-26T22:28:01.541137+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550090+0000","last_change":"2024-04-26T22:28:01.550090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550090+0000","last_undegraded":"2024-04-26T22:28:01.550090+0000","last_fullsized":"2024-04-26T22:28:01.550090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546936+0000","last_change":"2024-04-26T22:28:01.546936+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546936+0000","last_undegraded":"2024-04-26T22:28:01.546936+0000","last_fullsized":"2024-04-26T22:28:01.546936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543309+0000","last_change":"2024-04-26T22:28:01.543309+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543309+0000","last_undegraded":"2024-04-26T22:28:01.543309+0000","last_fullsized":"2024-04-26T22:28:01.543309+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543892+0000","last_change":"2024-04-26T22:28:01.543892+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543892+0000","last_undegraded":"2024-04-26T22:28:01.543892+0000","last_fullsized":"2024-04-26T22:28:01.543892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.554052+0000","last_change":"2024-04-26T22:28:01.554052+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.554052+0000","last_undegraded":"2024-04-26T22:28:01.554052+0000","last_fullsized":"2024-04-26T22:28:01.554052+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540104+0000","last_change":"2024-04-26T22:28:01.540104+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540104+0000","last_undegraded":"2024-04-26T22:28:01.540104+0000","last_fullsized":"2024-04-26T22:28:01.540104+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546355+0000","last_change":"2024-04-26T22:28:01.546355+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546355+0000","last_undegraded":"2024-04-26T22:28:01.546355+0000","last_fullsized":"2024-04-26T22:28:01.546355+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546942+0000","last_change":"2024-04-26T22:28:01.546942+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546942+0000","last_undegraded":"2024-04-26T22:28:01.546942+0000","last_fullsized":"2024-04-26T22:28:01.546942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552774+0000","last_change":"2024-04-26T22:28:01.552774+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552774+0000","last_undegraded":"2024-04-26T22:28:01.552774+0000","last_fullsized":"2024-04-26T22:28:01.552774+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.552113+0000","last_change":"2024-04-26T22:28:01.552113+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.552113+0000","last_undegraded":"2024-04-26T22:28:01.552113+0000","last_fullsized":"2024-04-26T22:28:01.552113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542939+0000","last_change":"2024-04-26T22:28:01.542939+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542939+0000","last_undegraded":"2024-04-26T22:28:01.542939+0000","last_fullsized":"2024-04-26T22:28:01.542939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543242+0000","last_change":"2024-04-26T22:28:01.543242+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543242+0000","last_undegraded":"2024-04-26T22:28:01.543242+0000","last_fullsized":"2024-04-26T22:28:01.543242+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540859+0000","last_change":"2024-04-26T22:28:01.540859+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540859+0000","last_undegraded":"2024-04-26T22:28:01.540859+0000","last_fullsized":"2024-04-26T22:28:01.540859+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546687+0000","last_change":"2024-04-26T22:28:01.546687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546687+0000","last_undegraded":"2024-04-26T22:28:01.546687+0000","last_fullsized":"2024-04-26T22:28:01.546687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540745+0000","last_change":"2024-04-26T22:28:01.540745+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540745+0000","last_undegraded":"2024-04-26T22:28:01.540745+0000","last_fullsized":"2024-04-26T22:28:01.540745+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550111+0000","last_change":"2024-04-26T22:28:01.550111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550111+0000","last_undegraded":"2024-04-26T22:28:01.550111+0000","last_fullsized":"2024-04-26T22:28:01.550111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551624+0000","last_change":"2024-04-26T22:28:01.551624+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551624+0000","last_undegraded":"2024-04-26T22:28:01.551624+0000","last_fullsized":"2024-04-26T22:28:01.551624+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550886+0000","last_change":"2024-04-26T22:28:01.550886+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550886+0000","last_undegraded":"2024-04-26T22:28:01.550886+0000","last_fullsized":"2024-04-26T22:28:01.550886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540303+0000","last_change":"2024-04-26T22:28:01.540303+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540303+0000","last_undegraded":"2024-04-26T22:28:01.540303+0000","last_fullsized":"2024-04-26T22:28:01.540303+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541775+0000","last_change":"2024-04-26T22:28:01.541775+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541775+0000","last_undegraded":"2024-04-26T22:28:01.541775+0000","last_fullsized":"2024-04-26T22:28:01.541775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546895+0000","last_change":"2024-04-26T22:28:01.546895+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546895+0000","last_undegraded":"2024-04-26T22:28:01.546895+0000","last_fullsized":"2024-04-26T22:28:01.546895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543109+0000","last_change":"2024-04-26T22:28:01.543109+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543109+0000","last_undegraded":"2024-04-26T22:28:01.543109+0000","last_fullsized":"2024-04-26T22:28:01.543109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550216+0000","last_change":"2024-04-26T22:28:01.550216+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550216+0000","last_undegraded":"2024-04-26T22:28:01.550216+0000","last_fullsized":"2024-04-26T22:28:01.550216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547102+0000","last_change":"2024-04-26T22:28:01.547102+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547102+0000","last_undegraded":"2024-04-26T22:28:01.547102+0000","last_fullsized":"2024-04-26T22:28:01.547102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554323+0000","last_change":"2024-04-26T22:28:01.551148+0000","last_active":"2024-04-26T22:28:01.554323+0000","last_peered":"2024-04-26T22:28:01.554323+0000","last_clean":"2024-04-26T22:28:01.554323+0000","last_became_active":"2024-04-26T22:28:01.550854+0000","last_became_peered":"2024-04-26T22:28:01.550854+0000","last_unstale":"2024-04-26T22:28:01.554323+0000","last_undegraded":"2024-04-26T22:28:01.554323+0000","last_fullsized":"2024-04-26T22:28:01.554323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543631+0000","last_change":"2024-04-26T22:28:01.543631+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543631+0000","last_undegraded":"2024-04-26T22:28:01.543631+0000","last_fullsized":"2024-04-26T22:28:01.543631+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556848+0000","last_change":"2024-04-26T22:28:01.553286+0000","last_active":"2024-04-26T22:28:01.556848+0000","last_peered":"2024-04-26T22:28:01.556848+0000","last_clean":"2024-04-26T22:28:01.556848+0000","last_became_active":"2024-04-26T22:28:01.552926+0000","last_became_peered":"2024-04-26T22:28:01.552926+0000","last_unstale":"2024-04-26T22:28:01.556848+0000","last_undegraded":"2024-04-26T22:28:01.556848+0000","last_fullsized":"2024-04-26T22:28:01.556848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541804+0000","last_change":"2024-04-26T22:28:01.541804+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541804+0000","last_undegraded":"2024-04-26T22:28:01.541804+0000","last_fullsized":"2024-04-26T22:28:01.541804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566207+0000","last_change":"2024-04-26T22:28:01.559844+0000","last_active":"2024-04-26T22:28:01.566207+0000","last_peered":"2024-04-26T22:28:01.566207+0000","last_clean":"2024-04-26T22:28:01.566207+0000","last_became_active":"2024-04-26T22:28:01.559546+0000","last_became_peered":"2024-04-26T22:28:01.559546+0000","last_unstale":"2024-04-26T22:28:01.566207+0000","last_undegraded":"2024-04-26T22:28:01.566207+0000","last_fullsized":"2024-04-26T22:28:01.566207+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:59.038151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550464+0000","last_change":"2024-04-26T22:28:01.550464+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550464+0000","last_undegraded":"2024-04-26T22:28:01.550464+0000","last_fullsized":"2024-04-26T22:28:01.550464+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554674+0000","last_change":"2024-04-26T22:28:01.552326+0000","last_active":"2024-04-26T22:28:01.554674+0000","last_peered":"2024-04-26T22:28:01.554674+0000","last_clean":"2024-04-26T22:28:01.554674+0000","last_became_active":"2024-04-26T22:28:01.551423+0000","last_became_peered":"2024-04-26T22:28:01.551423+0000","last_unstale":"2024-04-26T22:28:01.554674+0000","last_undegraded":"2024-04-26T22:28:01.554674+0000","last_fullsized":"2024-04-26T22:28:01.554674+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:17.520923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546338+0000","last_change":"2024-04-26T22:28:01.546338+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546338+0000","last_undegraded":"2024-04-26T22:28:01.546338+0000","last_fullsized":"2024-04-26T22:28:01.546338+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554857+0000","last_change":"2024-04-26T22:28:01.549390+0000","last_active":"2024-04-26T22:28:01.554857+0000","last_peered":"2024-04-26T22:28:01.554857+0000","last_clean":"2024-04-26T22:28:01.554857+0000","last_became_active":"2024-04-26T22:28:01.548063+0000","last_became_peered":"2024-04-26T22:28:01.548063+0000","last_unstale":"2024-04-26T22:28:01.554857+0000","last_undegraded":"2024-04-26T22:28:01.554857+0000","last_fullsized":"2024-04-26T22:28:01.554857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:20.079402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540460+0000","last_change":"2024-04-26T22:28:01.540460+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540460+0000","last_undegraded":"2024-04-26T22:28:01.540460+0000","last_fullsized":"2024-04-26T22:28:01.540460+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553031+0000","last_change":"2024-04-26T22:28:01.548404+0000","last_active":"2024-04-26T22:28:01.553031+0000","last_peered":"2024-04-26T22:28:01.553031+0000","last_clean":"2024-04-26T22:28:01.553031+0000","last_became_active":"2024-04-26T22:28:01.548088+0000","last_became_peered":"2024-04-26T22:28:01.548088+0000","last_unstale":"2024-04-26T22:28:01.553031+0000","last_undegraded":"2024-04-26T22:28:01.553031+0000","last_fullsized":"2024-04-26T22:28:01.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:38.650628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542687+0000","last_change":"2024-04-26T22:28:01.542687+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542687+0000","last_undegraded":"2024-04-26T22:28:01.542687+0000","last_fullsized":"2024-04-26T22:28:01.542687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563282+0000","last_change":"2024-04-26T22:28:01.556939+0000","last_active":"2024-04-26T22:28:01.563282+0000","last_peered":"2024-04-26T22:28:01.563282+0000","last_clean":"2024-04-26T22:28:01.563282+0000","last_became_active":"2024-04-26T22:28:01.556663+0000","last_became_peered":"2024-04-26T22:28:01.556663+0000","last_unstale":"2024-04-26T22:28:01.563282+0000","last_undegraded":"2024-04-26T22:28:01.563282+0000","last_fullsized":"2024-04-26T22:28:01.563282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549555+0000","last_change":"2024-04-26T22:28:01.549555+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549555+0000","last_undegraded":"2024-04-26T22:28:01.549555+0000","last_fullsized":"2024-04-26T22:28:01.549555+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554932+0000","last_change":"2024-04-26T22:28:01.552353+0000","last_active":"2024-04-26T22:28:01.554932+0000","last_peered":"2024-04-26T22:28:01.554932+0000","last_clean":"2024-04-26T22:28:01.554932+0000","last_became_active":"2024-04-26T22:28:01.549925+0000","last_became_peered":"2024-04-26T22:28:01.549925+0000","last_unstale":"2024-04-26T22:28:01.554932+0000","last_undegraded":"2024-04-26T22:28:01.554932+0000","last_fullsized":"2024-04-26T22:28:01.554932+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:13:42.686033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543786+0000","last_change":"2024-04-26T22:28:01.543786+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543786+0000","last_undegraded":"2024-04-26T22:28:01.543786+0000","last_fullsized":"2024-04-26T22:28:01.543786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558468+0000","last_change":"2024-04-26T22:28:01.555381+0000","last_active":"2024-04-26T22:28:01.558468+0000","last_peered":"2024-04-26T22:28:01.558468+0000","last_clean":"2024-04-26T22:28:01.558468+0000","last_became_active":"2024-04-26T22:28:01.555132+0000","last_became_peered":"2024-04-26T22:28:01.555132+0000","last_unstale":"2024-04-26T22:28:01.558468+0000","last_undegraded":"2024-04-26T22:28:01.558468+0000","last_fullsized":"2024-04-26T22:28:01.558468+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.553709+0000","last_change":"2024-04-26T22:28:01.553709+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.553709+0000","last_undegraded":"2024-04-26T22:28:01.553709+0000","last_fullsized":"2024-04-26T22:28:01.553709+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566486+0000","last_change":"2024-04-26T22:28:01.563856+0000","last_active":"2024-04-26T22:28:01.566486+0000","last_peered":"2024-04-26T22:28:01.566486+0000","last_clean":"2024-04-26T22:28:01.566486+0000","last_became_active":"2024-04-26T22:28:01.563514+0000","last_became_peered":"2024-04-26T22:28:01.563514+0000","last_unstale":"2024-04-26T22:28:01.566486+0000","last_undegraded":"2024-04-26T22:28:01.566486+0000","last_fullsized":"2024-04-26T22:28:01.566486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551090+0000","last_change":"2024-04-26T22:28:01.551090+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551090+0000","last_undegraded":"2024-04-26T22:28:01.551090+0000","last_fullsized":"2024-04-26T22:28:01.551090+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555437+0000","last_change":"2024-04-26T22:28:01.553958+0000","last_active":"2024-04-26T22:28:01.555437+0000","last_peered":"2024-04-26T22:28:01.555437+0000","last_clean":"2024-04-26T22:28:01.555437+0000","last_became_active":"2024-04-26T22:28:01.553552+0000","last_became_peered":"2024-04-26T22:28:01.553552+0000","last_unstale":"2024-04-26T22:28:01.555437+0000","last_undegraded":"2024-04-26T22:28:01.555437+0000","last_fullsized":"2024-04-26T22:28:01.555437+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540973+0000","last_change":"2024-04-26T22:28:01.540973+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540973+0000","last_undegraded":"2024-04-26T22:28:01.540973+0000","last_fullsized":"2024-04-26T22:28:01.540973+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566167+0000","last_change":"2024-04-26T22:28:01.559144+0000","last_active":"2024-04-26T22:28:01.566167+0000","last_peered":"2024-04-26T22:28:01.566167+0000","last_clean":"2024-04-26T22:28:01.566167+0000","last_became_active":"2024-04-26T22:28:01.558496+0000","last_became_peered":"2024-04-26T22:28:01.558496+0000","last_unstale":"2024-04-26T22:28:01.566167+0000","last_undegraded":"2024-04-26T22:28:01.566167+0000","last_fullsized":"2024-04-26T22:28:01.566167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.551099+0000","last_change":"2024-04-26T22:28:01.551099+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.551099+0000","last_undegraded":"2024-04-26T22:28:01.551099+0000","last_fullsized":"2024-04-26T22:28:01.551099+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554892+0000","last_change":"2024-04-26T22:28:01.550978+0000","last_active":"2024-04-26T22:28:01.554892+0000","last_peered":"2024-04-26T22:28:01.554892+0000","last_clean":"2024-04-26T22:28:01.554892+0000","last_became_active":"2024-04-26T22:28:01.550154+0000","last_became_peered":"2024-04-26T22:28:01.550154+0000","last_unstale":"2024-04-26T22:28:01.554892+0000","last_undegraded":"2024-04-26T22:28:01.554892+0000","last_fullsized":"2024-04-26T22:28:01.554892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:59.713113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.550436+0000","last_change":"2024-04-26T22:28:01.550436+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.550436+0000","last_undegraded":"2024-04-26T22:28:01.550436+0000","last_fullsized":"2024-04-26T22:28:01.550436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559137+0000","last_change":"2024-04-26T22:28:01.551899+0000","last_active":"2024-04-26T22:28:01.559137+0000","last_peered":"2024-04-26T22:28:01.559137+0000","last_clean":"2024-04-26T22:28:01.559137+0000","last_became_active":"2024-04-26T22:28:01.551587+0000","last_became_peered":"2024-04-26T22:28:01.551587+0000","last_unstale":"2024-04-26T22:28:01.559137+0000","last_undegraded":"2024-04-26T22:28:01.559137+0000","last_fullsized":"2024-04-26T22:28:01.559137+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:09.972664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549849+0000","last_change":"2024-04-26T22:28:01.549849+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549849+0000","last_undegraded":"2024-04-26T22:28:01.549849+0000","last_fullsized":"2024-04-26T22:28:01.549849+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558181+0000","last_change":"2024-04-26T22:28:01.555910+0000","last_active":"2024-04-26T22:28:01.558181+0000","last_peered":"2024-04-26T22:28:01.558181+0000","last_clean":"2024-04-26T22:28:01.558181+0000","last_became_active":"2024-04-26T22:28:01.555596+0000","last_became_peered":"2024-04-26T22:28:01.555596+0000","last_unstale":"2024-04-26T22:28:01.558181+0000","last_undegraded":"2024-04-26T22:28:01.558181+0000","last_fullsized":"2024-04-26T22:28:01.558181+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542675+0000","last_change":"2024-04-26T22:28:01.542675+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542675+0000","last_undegraded":"2024-04-26T22:28:01.542675+0000","last_fullsized":"2024-04-26T22:28:01.542675+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566270+0000","last_change":"2024-04-26T22:28:01.559695+0000","last_active":"2024-04-26T22:28:01.566270+0000","last_peered":"2024-04-26T22:28:01.566270+0000","last_clean":"2024-04-26T22:28:01.566270+0000","last_became_active":"2024-04-26T22:28:01.559448+0000","last_became_peered":"2024-04-26T22:28:01.559448+0000","last_unstale":"2024-04-26T22:28:01.566270+0000","last_undegraded":"2024-04-26T22:28:01.566270+0000","last_fullsized":"2024-04-26T22:28:01.566270+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542254+0000","last_change":"2024-04-26T22:28:01.542254+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542254+0000","last_undegraded":"2024-04-26T22:28:01.542254+0000","last_fullsized":"2024-04-26T22:28:01.542254+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.566076+0000","last_change":"2024-04-26T22:28:01.560134+0000","last_active":"2024-04-26T22:28:01.566076+0000","last_peered":"2024-04-26T22:28:01.566076+0000","last_clean":"2024-04-26T22:28:01.566076+0000","last_became_active":"2024-04-26T22:28:01.559832+0000","last_became_peered":"2024-04-26T22:28:01.559832+0000","last_unstale":"2024-04-26T22:28:01.566076+0000","last_undegraded":"2024-04-26T22:28:01.566076+0000","last_fullsized":"2024-04-26T22:28:01.566076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542106+0000","last_change":"2024-04-26T22:28:01.542106+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542106+0000","last_undegraded":"2024-04-26T22:28:01.542106+0000","last_fullsized":"2024-04-26T22:28:01.542106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554944+0000","last_change":"2024-04-26T22:28:01.550770+0000","last_active":"2024-04-26T22:28:01.554944+0000","last_peered":"2024-04-26T22:28:01.554944+0000","last_clean":"2024-04-26T22:28:01.554944+0000","last_became_active":"2024-04-26T22:28:01.550490+0000","last_became_peered":"2024-04-26T22:28:01.550490+0000","last_unstale":"2024-04-26T22:28:01.554944+0000","last_undegraded":"2024-04-26T22:28:01.554944+0000","last_fullsized":"2024-04-26T22:28:01.554944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:29.207017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549567+0000","last_change":"2024-04-26T22:28:01.549567+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549567+0000","last_undegraded":"2024-04-26T22:28:01.549567+0000","last_fullsized":"2024-04-26T22:28:01.549567+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.553033+0000","last_change":"2024-04-26T22:28:01.548123+0000","last_active":"2024-04-26T22:28:01.553033+0000","last_peered":"2024-04-26T22:28:01.553033+0000","last_clean":"2024-04-26T22:28:01.553033+0000","last_became_active":"2024-04-26T22:28:01.547847+0000","last_became_peered":"2024-04-26T22:28:01.547847+0000","last_unstale":"2024-04-26T22:28:01.553033+0000","last_undegraded":"2024-04-26T22:28:01.553033+0000","last_fullsized":"2024-04-26T22:28:01.553033+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:50.283072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543141+0000","last_change":"2024-04-26T22:28:01.543141+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543141+0000","last_undegraded":"2024-04-26T22:28:01.543141+0000","last_fullsized":"2024-04-26T22:28:01.543141+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554806+0000","last_change":"2024-04-26T22:28:01.548757+0000","last_active":"2024-04-26T22:28:01.554806+0000","last_peered":"2024-04-26T22:28:01.554806+0000","last_clean":"2024-04-26T22:28:01.554806+0000","last_became_active":"2024-04-26T22:28:01.548498+0000","last_became_peered":"2024-04-26T22:28:01.548498+0000","last_unstale":"2024-04-26T22:28:01.554806+0000","last_undegraded":"2024-04-26T22:28:01.554806+0000","last_fullsized":"2024-04-26T22:28:01.554806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:40:39.207881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543163+0000","last_change":"2024-04-26T22:28:01.543163+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543163+0000","last_undegraded":"2024-04-26T22:28:01.543163+0000","last_fullsized":"2024-04-26T22:28:01.543163+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.560904+0000","last_change":"2024-04-26T22:28:01.559890+0000","last_active":"2024-04-26T22:28:01.560904+0000","last_peered":"2024-04-26T22:28:01.560904+0000","last_clean":"2024-04-26T22:28:01.560904+0000","last_became_active":"2024-04-26T22:28:01.559564+0000","last_became_peered":"2024-04-26T22:28:01.559564+0000","last_unstale":"2024-04-26T22:28:01.560904+0000","last_undegraded":"2024-04-26T22:28:01.560904+0000","last_fullsized":"2024-04-26T22:28:01.560904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543175+0000","last_change":"2024-04-26T22:28:01.543175+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543175+0000","last_undegraded":"2024-04-26T22:28:01.543175+0000","last_fullsized":"2024-04-26T22:28:01.543175+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559529+0000","last_change":"2024-04-26T22:28:01.558972+0000","last_active":"2024-04-26T22:28:01.559529+0000","last_peered":"2024-04-26T22:28:01.559529+0000","last_clean":"2024-04-26T22:28:01.559529+0000","last_became_active":"2024-04-26T22:28:01.558693+0000","last_became_peered":"2024-04-26T22:28:01.558693+0000","last_unstale":"2024-04-26T22:28:01.559529+0000","last_undegraded":"2024-04-26T22:28:01.559529+0000","last_fullsized":"2024-04-26T22:28:01.559529+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.542997+0000","last_change":"2024-04-26T22:28:01.542997+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.542997+0000","last_undegraded":"2024-04-26T22:28:01.542997+0000","last_fullsized":"2024-04-26T22:28:01.542997+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555059+0000","last_change":"2024-04-26T22:28:01.551281+0000","last_active":"2024-04-26T22:28:01.555059+0000","last_peered":"2024-04-26T22:28:01.555059+0000","last_clean":"2024-04-26T22:28:01.555059+0000","last_became_active":"2024-04-26T22:28:01.550195+0000","last_became_peered":"2024-04-26T22:28:01.550195+0000","last_unstale":"2024-04-26T22:28:01.555059+0000","last_undegraded":"2024-04-26T22:28:01.555059+0000","last_fullsized":"2024-04-26T22:28:01.555059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:16:58.163201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.548111+0000","last_change":"2024-04-26T22:28:01.548111+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.548111+0000","last_undegraded":"2024-04-26T22:28:01.548111+0000","last_fullsized":"2024-04-26T22:28:01.548111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.555367+0000","last_change":"2024-04-26T22:28:01.553831+0000","last_active":"2024-04-26T22:28:01.555367+0000","last_peered":"2024-04-26T22:28:01.555367+0000","last_clean":"2024-04-26T22:28:01.555367+0000","last_became_active":"2024-04-26T22:28:01.553563+0000","last_became_peered":"2024-04-26T22:28:01.553563+0000","last_unstale":"2024-04-26T22:28:01.555367+0000","last_undegraded":"2024-04-26T22:28:01.555367+0000","last_fullsized":"2024-04-26T22:28:01.555367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543247+0000","last_change":"2024-04-26T22:28:01.543247+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543247+0000","last_undegraded":"2024-04-26T22:28:01.543247+0000","last_fullsized":"2024-04-26T22:28:01.543247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565921+0000","last_change":"2024-04-26T22:28:01.558893+0000","last_active":"2024-04-26T22:28:01.565921+0000","last_peered":"2024-04-26T22:28:01.565921+0000","last_clean":"2024-04-26T22:28:01.565921+0000","last_became_active":"2024-04-26T22:28:01.558541+0000","last_became_peered":"2024-04-26T22:28:01.558541+0000","last_unstale":"2024-04-26T22:28:01.565921+0000","last_undegraded":"2024-04-26T22:28:01.565921+0000","last_fullsized":"2024-04-26T22:28:01.565921+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.540152+0000","last_change":"2024-04-26T22:28:01.540152+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.540152+0000","last_undegraded":"2024-04-26T22:28:01.540152+0000","last_fullsized":"2024-04-26T22:28:01.540152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.563366+0000","last_change":"2024-04-26T22:28:01.560436+0000","last_active":"2024-04-26T22:28:01.563366+0000","last_peered":"2024-04-26T22:28:01.563366+0000","last_clean":"2024-04-26T22:28:01.563366+0000","last_became_active":"2024-04-26T22:28:01.560153+0000","last_became_peered":"2024-04-26T22:28:01.560153+0000","last_unstale":"2024-04-26T22:28:01.563366+0000","last_undegraded":"2024-04-26T22:28:01.563366+0000","last_fullsized":"2024-04-26T22:28:01.563366+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546676+0000","last_change":"2024-04-26T22:28:01.546676+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546676+0000","last_undegraded":"2024-04-26T22:28:01.546676+0000","last_fullsized":"2024-04-26T22:28:01.546676+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.565905+0000","last_change":"2024-04-26T22:28:01.558609+0000","last_active":"2024-04-26T22:28:01.565905+0000","last_peered":"2024-04-26T22:28:01.565905+0000","last_clean":"2024-04-26T22:28:01.565905+0000","last_became_active":"2024-04-26T22:28:01.558332+0000","last_became_peered":"2024-04-26T22:28:01.558332+0000","last_unstale":"2024-04-26T22:28:01.565905+0000","last_undegraded":"2024-04-26T22:28:01.565905+0000","last_fullsized":"2024-04-26T22:28:01.565905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.543092+0000","last_change":"2024-04-26T22:28:01.543092+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.543092+0000","last_undegraded":"2024-04-26T22:28:01.543092+0000","last_fullsized":"2024-04-26T22:28:01.543092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556844+0000","last_change":"2024-04-26T22:28:01.551370+0000","last_active":"2024-04-26T22:28:01.556844+0000","last_peered":"2024-04-26T22:28:01.556844+0000","last_clean":"2024-04-26T22:28:01.556844+0000","last_became_active":"2024-04-26T22:28:01.550977+0000","last_became_peered":"2024-04-26T22:28:01.550977+0000","last_unstale":"2024-04-26T22:28:01.556844+0000","last_undegraded":"2024-04-26T22:28:01.556844+0000","last_fullsized":"2024-04-26T22:28:01.556844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.546076+0000","last_change":"2024-04-26T22:28:01.546076+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.546076+0000","last_undegraded":"2024-04-26T22:28:01.546076+0000","last_fullsized":"2024-04-26T22:28:01.546076+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.559355+0000","last_change":"2024-04-26T22:28:01.555714+0000","last_active":"2024-04-26T22:28:01.559355+0000","last_peered":"2024-04-26T22:28:01.559355+0000","last_clean":"2024-04-26T22:28:01.559355+0000","last_became_active":"2024-04-26T22:28:01.555437+0000","last_became_peered":"2024-04-26T22:28:01.555437+0000","last_unstale":"2024-04-26T22:28:01.559355+0000","last_undegraded":"2024-04-26T22:28:01.559355+0000","last_fullsized":"2024-04-26T22:28:01.559355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.549204+0000","last_change":"2024-04-26T22:28:01.549204+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.549204+0000","last_undegraded":"2024-04-26T22:28:01.549204+0000","last_fullsized":"2024-04-26T22:28:01.549204+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.554873+0000","last_change":"2024-04-26T22:28:01.549998+0000","last_active":"2024-04-26T22:28:01.554873+0000","last_peered":"2024-04-26T22:28:01.554873+0000","last_clean":"2024-04-26T22:28:01.554873+0000","last_became_active":"2024-04-26T22:28:01.549526+0000","last_became_peered":"2024-04-26T22:28:01.549526+0000","last_unstale":"2024-04-26T22:28:01.554873+0000","last_undegraded":"2024-04-26T22:28:01.554873+0000","last_fullsized":"2024-04-26T22:28:01.554873+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:28:36.732304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.541224+0000","last_change":"2024-04-26T22:28:01.541224+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.541224+0000","last_undegraded":"2024-04-26T22:28:01.541224+0000","last_fullsized":"2024-04-26T22:28:01.541224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.558246+0000","last_change":"2024-04-26T22:28:01.555358+0000","last_active":"2024-04-26T22:28:01.558246+0000","last_peered":"2024-04-26T22:28:01.558246+0000","last_clean":"2024-04-26T22:28:01.558246+0000","last_became_active":"2024-04-26T22:28:01.555056+0000","last_became_peered":"2024-04-26T22:28:01.555056+0000","last_unstale":"2024-04-26T22:28:01.558246+0000","last_undegraded":"2024-04-26T22:28:01.558246+0000","last_fullsized":"2024-04-26T22:28:01.558246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-04-26T22:28:01.547161+0000","last_change":"2024-04-26T22:28:01.547161+0000","last_active":"2024-04-26T22:28:01.529467+0000","last_peered":"2024-04-26T22:28:01.529467+0000","last_clean":"2024-04-26T22:28:01.529467+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:01.547161+0000","last_undegraded":"2024-04-26T22:28:01.547161+0000","last_fullsized":"2024-04-26T22:28:01.547161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.556304+0000","last_change":"2024-04-26T22:28:01.554918+0000","last_active":"2024-04-26T22:28:01.556304+0000","last_peered":"2024-04-26T22:28:01.556304+0000","last_clean":"2024-04-26T22:28:01.556304+0000","last_became_active":"2024-04-26T22:28:01.554637+0000","last_became_peered":"2024-04-26T22:28:01.554637+0000","last_unstale":"2024-04-26T22:28:01.556304+0000","last_undegraded":"2024-04-26T22:28:01.556304+0000","last_fullsized":"2024-04-26T22:28:01.556304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T22:28:01.550940+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:01.550940+0000","last_peered":"2024-04-26T22:28:01.550940+0000","last_clean":"2024-04-26T22:28:01.550940+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:01.550940+0000","last_undegraded":"2024-04-26T22:28:01.550940+0000","last_fullsized":"2024-04-26T22:28:01.550940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":115349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.544043+0000', 'last_change': '2024-04-26T22:28:01.544043+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.544043+0000', 'last_undegraded': '2024-04-26T22:28:01.544043+0000', 'last_fullsized': '2024-04-26T22:28:01.544043+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.555306+0000', 'last_change': '2024-04-26T22:28:01.555306+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.555306+0000', 'last_undegraded': '2024-04-26T22:28:01.555306+0000', 'last_fullsized': '2024-04-26T22:28:01.555306+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540215+0000', 'last_change': '2024-04-26T22:28:01.540215+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540215+0000', 'last_undegraded': '2024-04-26T22:28:01.540215+0000', 'last_fullsized': '2024-04-26T22:28:01.540215+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550106+0000', 'last_change': '2024-04-26T22:28:01.550106+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550106+0000', 'last_undegraded': '2024-04-26T22:28:01.550106+0000', 'last_fullsized': '2024-04-26T22:28:01.550106+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.544326+0000', 'last_change': '2024-04-26T22:28:01.544326+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.544326+0000', 'last_undegraded': '2024-04-26T22:28:01.544326+0000', 'last_fullsized': '2024-04-26T22:28:01.544326+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.554580+0000', 'last_change': '2024-04-26T22:28:01.554580+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.554580+0000', 'last_undegraded': '2024-04-26T22:28:01.554580+0000', 'last_fullsized': '2024-04-26T22:28:01.554580+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.547702+0000', 'last_change': '2024-04-26T22:28:01.547702+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.547702+0000', 'last_undegraded': '2024-04-26T22:28:01.547702+0000', 'last_fullsized': '2024-04-26T22:28:01.547702+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550612+0000', 'last_change': '2024-04-26T22:28:01.550612+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550612+0000', 'last_undegraded': '2024-04-26T22:28:01.550612+0000', 'last_fullsized': '2024-04-26T22:28:01.550612+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-26T22:28:07.279 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.541137+0000', 'last_change': '2024-04-26T22:28:01.541137+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.541137+0000', 'last_undegraded': '2024-04-26T22:28:01.541137+0000', 'last_fullsized': '2024-04-26T22:28:01.541137+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550090+0000', 'last_change': '2024-04-26T22:28:01.550090+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550090+0000', 'last_undegraded': '2024-04-26T22:28:01.550090+0000', 'last_fullsized': '2024-04-26T22:28:01.550090+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546936+0000', 'last_change': '2024-04-26T22:28:01.546936+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546936+0000', 'last_undegraded': '2024-04-26T22:28:01.546936+0000', 'last_fullsized': '2024-04-26T22:28:01.546936+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543309+0000', 'last_change': '2024-04-26T22:28:01.543309+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543309+0000', 'last_undegraded': '2024-04-26T22:28:01.543309+0000', 'last_fullsized': '2024-04-26T22:28:01.543309+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543892+0000', 'last_change': '2024-04-26T22:28:01.543892+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543892+0000', 'last_undegraded': '2024-04-26T22:28:01.543892+0000', 'last_fullsized': '2024-04-26T22:28:01.543892+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.554052+0000', 'last_change': '2024-04-26T22:28:01.554052+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.554052+0000', 'last_undegraded': '2024-04-26T22:28:01.554052+0000', 'last_fullsized': '2024-04-26T22:28:01.554052+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-26T22:28:07.280 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540104+0000', 'last_change': '2024-04-26T22:28:01.540104+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540104+0000', 'last_undegraded': '2024-04-26T22:28:01.540104+0000', 'last_fullsized': '2024-04-26T22:28:01.540104+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546355+0000', 'last_change': '2024-04-26T22:28:01.546355+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546355+0000', 'last_undegraded': '2024-04-26T22:28:01.546355+0000', 'last_fullsized': '2024-04-26T22:28:01.546355+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546942+0000', 'last_change': '2024-04-26T22:28:01.546942+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546942+0000', 'last_undegraded': '2024-04-26T22:28:01.546942+0000', 'last_fullsized': '2024-04-26T22:28:01.546942+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.552774+0000', 'last_change': '2024-04-26T22:28:01.552774+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.552774+0000', 'last_undegraded': '2024-04-26T22:28:01.552774+0000', 'last_fullsized': '2024-04-26T22:28:01.552774+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.552113+0000', 'last_change': '2024-04-26T22:28:01.552113+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.552113+0000', 'last_undegraded': '2024-04-26T22:28:01.552113+0000', 'last_fullsized': '2024-04-26T22:28:01.552113+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542939+0000', 'last_change': '2024-04-26T22:28:01.542939+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542939+0000', 'last_undegraded': '2024-04-26T22:28:01.542939+0000', 'last_fullsized': '2024-04-26T22:28:01.542939+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.281 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543242+0000', 'last_change': '2024-04-26T22:28:01.543242+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543242+0000', 'last_undegraded': '2024-04-26T22:28:01.543242+0000', 'last_fullsized': '2024-04-26T22:28:01.543242+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540859+0000', 'last_change': '2024-04-26T22:28:01.540859+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540859+0000', 'last_undegraded': '2024-04-26T22:28:01.540859+0000', 'last_fullsized': '2024-04-26T22:28:01.540859+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546687+0000', 'last_change': '2024-04-26T22:28:01.546687+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546687+0000', 'last_undegraded': '2024-04-26T22:28:01.546687+0000', 'last_fullsized': '2024-04-26T22:28:01.546687+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540745+0000', 'last_change': '2024-04-26T22:28:01.540745+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540745+0000', 'last_undegraded': '2024-04-26T22:28:01.540745+0000', 'last_fullsized': '2024-04-26T22:28:01.540745+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550111+0000', 'last_change': '2024-04-26T22:28:01.550111+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550111+0000', 'last_undegraded': '2024-04-26T22:28:01.550111+0000', 'last_fullsized': '2024-04-26T22:28:01.550111+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-26T22:28:07.282 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.551624+0000', 'last_change': '2024-04-26T22:28:01.551624+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.551624+0000', 'last_undegraded': '2024-04-26T22:28:01.551624+0000', 'last_fullsized': '2024-04-26T22:28:01.551624+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550886+0000', 'last_change': '2024-04-26T22:28:01.550886+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550886+0000', 'last_undegraded': '2024-04-26T22:28:01.550886+0000', 'last_fullsized': '2024-04-26T22:28:01.550886+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540303+0000', 'last_change': '2024-04-26T22:28:01.540303+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540303+0000', 'last_undegraded': '2024-04-26T22:28:01.540303+0000', 'last_fullsized': '2024-04-26T22:28:01.540303+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.541775+0000', 'last_change': '2024-04-26T22:28:01.541775+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.541775+0000', 'last_undegraded': '2024-04-26T22:28:01.541775+0000', 'last_fullsized': '2024-04-26T22:28:01.541775+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546895+0000', 'last_change': '2024-04-26T22:28:01.546895+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546895+0000', 'last_undegraded': '2024-04-26T22:28:01.546895+0000', 'last_fullsized': '2024-04-26T22:28:01.546895+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543109+0000', 'last_change': '2024-04-26T22:28:01.543109+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543109+0000', 'last_undegraded': '2024-04-26T22:28:01.543109+0000', 'last_fullsized': '2024-04-26T22:28:01.543109+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:07.283 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550216+0000', 'last_change': '2024-04-26T22:28:01.550216+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550216+0000', 'last_undegraded': '2024-04-26T22:28:01.550216+0000', 'last_fullsized': '2024-04-26T22:28:01.550216+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.547102+0000', 'last_change': '2024-04-26T22:28:01.547102+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.547102+0000', 'last_undegraded': '2024-04-26T22:28:01.547102+0000', 'last_fullsized': '2024-04-26T22:28:01.547102+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543631+0000', 'last_change': '2024-04-26T22:28:01.543631+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543631+0000', 'last_undegraded': '2024-04-26T22:28:01.543631+0000', 'last_fullsized': '2024-04-26T22:28:01.543631+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.541804+0000', 'last_change': '2024-04-26T22:28:01.541804+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.541804+0000', 'last_undegraded': '2024-04-26T22:28:01.541804+0000', 'last_fullsized': '2024-04-26T22:28:01.541804+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550464+0000', 'last_change': '2024-04-26T22:28:01.550464+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550464+0000', 'last_undegraded': '2024-04-26T22:28:01.550464+0000', 'last_fullsized': '2024-04-26T22:28:01.550464+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546338+0000', 'last_change': '2024-04-26T22:28:01.546338+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546338+0000', 'last_undegraded': '2024-04-26T22:28:01.546338+0000', 'last_fullsized': '2024-04-26T22:28:01.546338+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.284 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540460+0000', 'last_change': '2024-04-26T22:28:01.540460+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540460+0000', 'last_undegraded': '2024-04-26T22:28:01.540460+0000', 'last_fullsized': '2024-04-26T22:28:01.540460+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542687+0000', 'last_change': '2024-04-26T22:28:01.542687+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542687+0000', 'last_undegraded': '2024-04-26T22:28:01.542687+0000', 'last_fullsized': '2024-04-26T22:28:01.542687+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.549555+0000', 'last_change': '2024-04-26T22:28:01.549555+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.549555+0000', 'last_undegraded': '2024-04-26T22:28:01.549555+0000', 'last_fullsized': '2024-04-26T22:28:01.549555+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543786+0000', 'last_change': '2024-04-26T22:28:01.543786+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543786+0000', 'last_undegraded': '2024-04-26T22:28:01.543786+0000', 'last_fullsized': '2024-04-26T22:28:01.543786+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.553709+0000', 'last_change': '2024-04-26T22:28:01.553709+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.553709+0000', 'last_undegraded': '2024-04-26T22:28:01.553709+0000', 'last_fullsized': '2024-04-26T22:28:01.553709+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.551090+0000', 'last_change': '2024-04-26T22:28:01.551090+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.551090+0000', 'last_undegraded': '2024-04-26T22:28:01.551090+0000', 'last_fullsized': '2024-04-26T22:28:01.551090+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-26T22:28:07.285 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540973+0000', 'last_change': '2024-04-26T22:28:01.540973+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540973+0000', 'last_undegraded': '2024-04-26T22:28:01.540973+0000', 'last_fullsized': '2024-04-26T22:28:01.540973+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.551099+0000', 'last_change': '2024-04-26T22:28:01.551099+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.551099+0000', 'last_undegraded': '2024-04-26T22:28:01.551099+0000', 'last_fullsized': '2024-04-26T22:28:01.551099+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.550436+0000', 'last_change': '2024-04-26T22:28:01.550436+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.550436+0000', 'last_undegraded': '2024-04-26T22:28:01.550436+0000', 'last_fullsized': '2024-04-26T22:28:01.550436+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.549849+0000', 'last_change': '2024-04-26T22:28:01.549849+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.549849+0000', 'last_undegraded': '2024-04-26T22:28:01.549849+0000', 'last_fullsized': '2024-04-26T22:28:01.549849+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542675+0000', 'last_change': '2024-04-26T22:28:01.542675+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542675+0000', 'last_undegraded': '2024-04-26T22:28:01.542675+0000', 'last_fullsized': '2024-04-26T22:28:01.542675+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542254+0000', 'last_change': '2024-04-26T22:28:01.542254+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542254+0000', 'last_undegraded': '2024-04-26T22:28:01.542254+0000', 'last_fullsized': '2024-04-26T22:28:01.542254+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-26T22:28:07.286 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542106+0000', 'last_change': '2024-04-26T22:28:01.542106+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542106+0000', 'last_undegraded': '2024-04-26T22:28:01.542106+0000', 'last_fullsized': '2024-04-26T22:28:01.542106+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.549567+0000', 'last_change': '2024-04-26T22:28:01.549567+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.549567+0000', 'last_undegraded': '2024-04-26T22:28:01.549567+0000', 'last_fullsized': '2024-04-26T22:28:01.549567+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543141+0000', 'last_change': '2024-04-26T22:28:01.543141+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543141+0000', 'last_undegraded': '2024-04-26T22:28:01.543141+0000', 'last_fullsized': '2024-04-26T22:28:01.543141+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543163+0000', 'last_change': '2024-04-26T22:28:01.543163+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543163+0000', 'last_undegraded': '2024-04-26T22:28:01.543163+0000', 'last_fullsized': '2024-04-26T22:28:01.543163+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543175+0000', 'last_change': '2024-04-26T22:28:01.543175+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543175+0000', 'last_undegraded': '2024-04-26T22:28:01.543175+0000', 'last_fullsized': '2024-04-26T22:28:01.543175+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.542997+0000', 'last_change': '2024-04-26T22:28:01.542997+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.542997+0000', 'last_undegraded': '2024-04-26T22:28:01.542997+0000', 'last_fullsized': '2024-04-26T22:28:01.542997+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.548111+0000', 'last_change': '2024-04-26T22:28:01.548111+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.548111+0000', 'last_undegraded': '2024-04-26T22:28:01.548111+0000', 'last_fullsized': '2024-04-26T22:28:01.548111+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.287 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543247+0000', 'last_change': '2024-04-26T22:28:01.543247+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543247+0000', 'last_undegraded': '2024-04-26T22:28:01.543247+0000', 'last_fullsized': '2024-04-26T22:28:01.543247+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.540152+0000', 'last_change': '2024-04-26T22:28:01.540152+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.540152+0000', 'last_undegraded': '2024-04-26T22:28:01.540152+0000', 'last_fullsized': '2024-04-26T22:28:01.540152+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546676+0000', 'last_change': '2024-04-26T22:28:01.546676+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546676+0000', 'last_undegraded': '2024-04-26T22:28:01.546676+0000', 'last_fullsized': '2024-04-26T22:28:01.546676+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.543092+0000', 'last_change': '2024-04-26T22:28:01.543092+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.543092+0000', 'last_undegraded': '2024-04-26T22:28:01.543092+0000', 'last_fullsized': '2024-04-26T22:28:01.543092+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.546076+0000', 'last_change': '2024-04-26T22:28:01.546076+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.546076+0000', 'last_undegraded': '2024-04-26T22:28:01.546076+0000', 'last_fullsized': '2024-04-26T22:28:01.546076+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.549204+0000', 'last_change': '2024-04-26T22:28:01.549204+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.549204+0000', 'last_undegraded': '2024-04-26T22:28:01.549204+0000', 'last_fullsized': '2024-04-26T22:28:01.549204+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:07.288 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-26T22:28:07.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.541224+0000', 'last_change': '2024-04-26T22:28:01.541224+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.541224+0000', 'last_undegraded': '2024-04-26T22:28:01.541224+0000', 'last_fullsized': '2024-04-26T22:28:01.541224+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:07.289 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-26T22:28:07.289 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:01.547161+0000', 'last_change': '2024-04-26T22:28:01.547161+0000', 'last_active': '2024-04-26T22:28:01.529467+0000', 'last_peered': '2024-04-26T22:28:01.529467+0000', 'last_clean': '2024-04-26T22:28:01.529467+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:01.547161+0000', 'last_undegraded': '2024-04-26T22:28:01.547161+0000', 'last_fullsized': '2024-04-26T22:28:01.547161+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:01.529467+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:07.289 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T22:28:07.349 INFO:tasks.ceph.mds.a.smithi062.stderr:2024-04-26T22:28:07.349+0000 7fc11b108640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:28:07.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.415+0000 7f8a16ad1640 1 Processor -- start 2024-04-26T22:28:07.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.415+0000 7f8a16ad1640 1 -- start start 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a16ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a1016a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a16ad1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a100d2ed0 con 0x7f8a1015f470 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a16ad1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a100d31b0 con 0x7f8a100d0f00 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a16ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a100d3490 con 0x7f8a100d21b0 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a1486d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f8a1015f470 0x7f8a100d07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42362/0 (socket says 172.21.15.62:42362) 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a1486d640 1 -- 172.21.15.62:0/1843225359 learned_addr learned my addr 172.21.15.62:0/1843225359 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a1506e640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a1016a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1231033678 0 0) 0x7f8a100d2ed0 con 0x7f8a1015f470 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a00003520 con 0x7f8a1015f470 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 377893972 0 0) 0x7f8a00003520 con 0x7f8a1015f470 2024-04-26T22:28:07.418 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 msgr2=0x7f8a1016a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a1016a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 >> v1:172.21.15.174:6789/0 conn(0x7f8a100d0f00 legacy=0x7f8a100d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a100d3770 con 0x7f8a1015f470 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f8a04001670 con 0x7f8a1015f470 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 >> v1:172.21.15.62:6789/0 conn(0x7f8a1015f470 legacy=0x7f8a100d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a000039e0 0x7f8a00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a00004390 0x7f8a00004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a000050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100d3490 con 0x7f8a000039e0 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0000b9f0 con 0x7f8a00004390 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.416+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0000bbf0 con 0x7f8a100d21b0 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a000039e0 0x7f8a00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a0ffff640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a00004390 0x7f8a00004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1506e640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a000050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.419 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 -- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 msgr2=0x7f8a000050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a000050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 -- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a00004390 msgr2=0x7f8a00004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a00004390 0x7f8a00004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1486d640 1 -- 172.21.15.62:0/1843225359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a00003520 con 0x7f8a000039e0 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.417+0000 7f8a1506e640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a100d21b0 0x7f8a000050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a1486d640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a000039e0 0x7f8a00003dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c240 tx=0x7f8a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff2f0890d6a2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402e070 con 0x7f8a000039e0 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04005af0 con 0x7f8a000039e0 2024-04-26T22:28:07.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a0f7fe640 1 -- 172.21.15.62:0/1843225359 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04007d30 con 0x7f8a000039e0 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 -- 172.21.15.62:0/1843225359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a000039e0 msgr2=0x7f8a00003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/1843225359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a000039e0 0x7f8a00003dc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c240 tx=0x7f8a00005830 comp rx=0 tx=0).stop 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a1506e640 1 -- 172.21.15.62:0/1843225359 reap_dead start 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 -- 172.21.15.62:0/1843225359 shutdown_connections 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 -- 172.21.15.62:0/1843225359 >> 172.21.15.62:0/1843225359 conn(0x7f8a1009e510 msgr2=0x7f8a100d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 -- 172.21.15.62:0/1843225359 shutdown_connections 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.418+0000 7f8a16ad1640 1 -- 172.21.15.62:0/1843225359 wait complete. 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.419+0000 7f8a16ad1640 1 Processor -- start 2024-04-26T22:28:07.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.419+0000 7f8a16ad1640 1 -- start start 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a000039e0 0x7f8a100ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 0x7f8a100ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100d3f70 con 0x7f8a1014c1d0 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1015f850 con 0x7f8a100cdc70 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a16ad1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1014cb70 con 0x7f8a000039e0 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a0ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43376/0 (socket says 172.21.15.62:43376) 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.420+0000 7f8a0ffff640 1 -- 172.21.15.62:0/2047823753 learned_addr learned my addr 172.21.15.62:0/2047823753 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a1486d640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a000039e0 0x7f8a100ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a1506e640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 0x7f8a100ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 -- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a000039e0 msgr2=0x7f8a100ba0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a000039e0 0x7f8a100ba0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 -- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 msgr2=0x7f8a100ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 0x7f8a100ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 -- 172.21.15.62:0/2047823753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a101f96a0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a1506e640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 0x7f8a100ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0ffff640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00f780 tx=0x7f89fc00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00fe90 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc018070 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a101f88e0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00d730 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.421+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101f8ea0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.422+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1006a360 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.423+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89fc00d8d0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.423+0000 7f8a0d7fa640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 0x7f89f406ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.423+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f89fc08b9c0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.423+0000 7f8a1486d640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 0x7f89f406ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.424+0000 7f8a1486d640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 0x7f89f406ef50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a04033000 tx=0x7f8a0402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.427+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89fc08de20 con 0x7f8a1014c1d0 2024-04-26T22:28:07.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.686+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8a100d5870 con 0x7f8a1014c1d0 2024-04-26T22:28:07.686 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.686+0000 7f8a0d7fa640 1 -- 172.21.15.62:0/2047823753 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1581 (secure 0 0 0) 0x7f89fc059df0 con 0x7f8a1014c1d0 2024-04-26T22:28:07.687 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:07.687 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":31,"num_osds":8,"num_up_osds":8,"osd_up_since":1714170420,"num_in_osds":8,"osd_in_since":1714170411,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225042432,"bytes_avail":772869070848,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4837}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-26T22:28:03.651029+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.688+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 msgr2=0x7f89f406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.688+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 0x7f89f406ef50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a04033000 tx=0x7f8a0402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.688+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 msgr2=0x7f8a1014c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.688+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00f780 tx=0x7f89fc00fc40 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 shutdown_connections 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f89f406ca90 0x7f89f406ef50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a100cdc70 0x7f8a100ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8a1014c1d0 0x7f8a1014c630 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 --2- 172.21.15.62:0/2047823753 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a000039e0 0x7f8a100ba0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 >> 172.21.15.62:0/2047823753 conn(0x7f8a1009e510 msgr2=0x7f8a10075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:07.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 shutdown_connections 2024-04-26T22:28:07.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.689+0000 7f8a16ad1640 1 -- 172.21.15.62:0/2047823753 wait complete. 2024-04-26T22:28:07.699 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T22:28:07.700 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:07.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.840+0000 7f683795f640 1 Processor -- start 2024-04-26T22:28:07.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.840+0000 7f683795f640 1 -- start start 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f683795f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68300d3820 0x7f68300d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f683795f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 0x7f68300b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f683795f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68300d0210 con 0x7f6830162b40 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f683795f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f68300d41c0 con 0x7f68300d3820 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f683795f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f68300d04f0 con 0x7f68300b4ba0 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f6834efa640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f68300d3820 0x7f68300d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f68356fb640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6830162b40 0x7f68300d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42370/0 (socket says 172.21.15.62:42370) 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.841+0000 7f68356fb640 1 -- 172.21.15.62:0/500379423 learned_addr learned my addr 172.21.15.62:0/500379423 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6834efa640 1 -- 172.21.15.62:0/500379423 >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 msgr2=0x7f68300b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6835efc640 1 --2- 172.21.15.62:0/500379423 >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 0x7f68300b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6834efa640 1 --2- 172.21.15.62:0/500379423 >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 0x7f68300b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6834efa640 1 -- 172.21.15.62:0/500379423 >> v1:172.21.15.62:6789/0 conn(0x7f6830162b40 legacy=0x7f68300d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f682ffff640 1 -- 172.21.15.62:0/500379423 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762616381 0 0) 0x7f68300d0210 con 0x7f6830162b40 2024-04-26T22:28:07.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6834efa640 1 -- 172.21.15.62:0/500379423 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68300d07d0 con 0x7f68300d3820 2024-04-26T22:28:07.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6835efc640 1 --2- 172.21.15.62:0/500379423 >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 0x7f68300b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.842+0000 7f6834efa640 1 --2- 172.21.15.62:0/500379423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300d3820 0x7f68300d3c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f683009ec60 tx=0x7f682402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d353ec6a083ffde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f682ffff640 1 -- 172.21.15.62:0/500379423 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824037070 con 0x7f68300d3820 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f682ffff640 1 -- 172.21.15.62:0/500379423 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6824032440 con 0x7f68300d3820 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f682ffff640 1 -- 172.21.15.62:0/500379423 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824032810 con 0x7f68300d3820 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 -- 172.21.15.62:0/500379423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300d3820 msgr2=0x7f68300d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 --2- 172.21.15.62:0/500379423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300d3820 0x7f68300d3c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f683009ec60 tx=0x7f682402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 -- 172.21.15.62:0/500379423 shutdown_connections 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 --2- 172.21.15.62:0/500379423 >> v2:172.21.15.174:3300/0 conn(0x7f68300b4ba0 0x7f68300b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 --2- 172.21.15.62:0/500379423 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300d3820 0x7f68300d3c80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 -- 172.21.15.62:0/500379423 >> 172.21.15.62:0/500379423 conn(0x7f683009e550 msgr2=0x7f68300d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:07.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 -- 172.21.15.62:0/500379423 shutdown_connections 2024-04-26T22:28:07.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.843+0000 7f683795f640 1 -- 172.21.15.62:0/500379423 wait complete. 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.844+0000 7f683795f640 1 Processor -- start 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.844+0000 7f683795f640 1 -- start start 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 0x7f68300cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830162b40 0x7f68300ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f68300d12b0 con 0x7f68300b4ba0 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68300d0d90 con 0x7f6830162b40 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f683795f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f68300cfe30 con 0x7f68300d3820 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f68356fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f68356fb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43390/0 (socket says 172.21.15.62:43390) 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f68356fb640 1 -- 172.21.15.62:0/3440371645 learned_addr learned my addr 172.21.15.62:0/3440371645 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f6835efc640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830162b40 0x7f68300ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.845+0000 7f6834efa640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 0x7f68300cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 -- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 msgr2=0x7f68300cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 0x7f68300cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 -- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830162b40 msgr2=0x7f68300ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830162b40 0x7f68300ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 -- 172.21.15.62:0/3440371645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68300c9c40 con 0x7f68300b4ba0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f6834efa640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 0x7f68300cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f68356fb640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f682800c4d0 tx=0x7f682800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828018070 con 0x7f68300b4ba0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828007500 con 0x7f68300b4ba0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68300b8c90 con 0x7f68300b4ba0 2024-04-26T22:28:07.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828004a20 con 0x7f68300b4ba0 2024-04-26T22:28:07.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.846+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68300b9250 con 0x7f68300b4ba0 2024-04-26T22:28:07.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.847+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683006a360 con 0x7f68300b4ba0 2024-04-26T22:28:07.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.847+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f682801b070 con 0x7f68300b4ba0 2024-04-26T22:28:07.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.848+0000 7f682dffb640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:07.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.848+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f682808ade0 con 0x7f68300b4ba0 2024-04-26T22:28:07.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.848+0000 7f6834efa640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:07.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.849+0000 7f6834efa640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6824034000 tx=0x7f682402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:07.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:07.852+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6828058bc0 con 0x7f68300b4ba0 2024-04-26T22:28:08.033 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.033+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68300b67c0 con 0x7f681c06c8f0 2024-04-26T22:28:08.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.040+0000 7f682dffb640 1 -- 172.21.15.62:0/3440371645 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f68300b67c0 con 0x7f681c06c8f0 2024-04-26T22:28:08.042 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:08.044 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 msgr2=0x7f681c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 0x7f681c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6824034000 tx=0x7f682402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 msgr2=0x7f683015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f682800c4d0 tx=0x7f682800c990 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 shutdown_connections 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f681c06c8f0 0x7f681c06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830162b40 0x7f68300ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68300d3820 0x7f68300cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 --2- 172.21.15.62:0/3440371645 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f68300b4ba0 0x7f683015aa50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:08.045 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 >> 172.21.15.62:0/3440371645 conn(0x7f683009e550 msgr2=0x7f68300d2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:08.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 shutdown_connections 2024-04-26T22:28:08.046 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:08.043+0000 7f683795f640 1 -- 172.21.15.62:0/3440371645 wait complete. 2024-04-26T22:28:08.053 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-04-26T22:28:07.651858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220136,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754584,"statfs":{"total":773094113280,"available":772868694016,"internally_reserved":0,"allocated":4038656,"data_stored":2263084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.113620"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093321+0000","last_change":"2024-04-26T22:28:02.557853+0000","last_active":"2024-04-26T22:28:04.093321+0000","last_peered":"2024-04-26T22:28:04.093321+0000","last_clean":"2024-04-26T22:28:04.093321+0000","last_became_active":"2024-04-26T22:28:02.557547+0000","last_became_peered":"2024-04-26T22:28:02.557547+0000","last_unstale":"2024-04-26T22:28:04.093321+0000","last_undegraded":"2024-04-26T22:28:04.093321+0000","last_fullsized":"2024-04-26T22:28:04.093321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296667+0000","last_change":"2024-04-26T22:28:02.565136+0000","last_active":"2024-04-26T22:28:04.296667+0000","last_peered":"2024-04-26T22:28:04.296667+0000","last_clean":"2024-04-26T22:28:04.296667+0000","last_became_active":"2024-04-26T22:28:02.564862+0000","last_became_peered":"2024-04-26T22:28:02.564862+0000","last_unstale":"2024-04-26T22:28:04.296667+0000","last_undegraded":"2024-04-26T22:28:04.296667+0000","last_fullsized":"2024-04-26T22:28:04.296667+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:40.654516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561035+0000","last_change":"2024-04-26T22:28:02.568096+0000","last_active":"2024-04-26T22:28:03.561035+0000","last_peered":"2024-04-26T22:28:03.561035+0000","last_clean":"2024-04-26T22:28:03.561035+0000","last_became_active":"2024-04-26T22:28:02.567788+0000","last_became_peered":"2024-04-26T22:28:02.567788+0000","last_unstale":"2024-04-26T22:28:03.561035+0000","last_undegraded":"2024-04-26T22:28:03.561035+0000","last_fullsized":"2024-04-26T22:28:03.561035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:07.397727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559548+0000","last_change":"2024-04-26T22:28:02.563172+0000","last_active":"2024-04-26T22:28:03.559548+0000","last_peered":"2024-04-26T22:28:03.559548+0000","last_clean":"2024-04-26T22:28:03.559548+0000","last_became_active":"2024-04-26T22:28:02.562479+0000","last_became_peered":"2024-04-26T22:28:02.562479+0000","last_unstale":"2024-04-26T22:28:03.559548+0000","last_undegraded":"2024-04-26T22:28:03.559548+0000","last_fullsized":"2024-04-26T22:28:03.559548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914867+0000","last_change":"2024-04-26T22:28:02.565256+0000","last_active":"2024-04-26T22:28:03.914867+0000","last_peered":"2024-04-26T22:28:03.914867+0000","last_clean":"2024-04-26T22:28:03.914867+0000","last_became_active":"2024-04-26T22:28:02.564971+0000","last_became_peered":"2024-04-26T22:28:02.564971+0000","last_unstale":"2024-04-26T22:28:03.914867+0000","last_undegraded":"2024-04-26T22:28:03.914867+0000","last_fullsized":"2024-04-26T22:28:03.914867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:10.979401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296114+0000","last_change":"2024-04-26T22:28:02.562239+0000","last_active":"2024-04-26T22:28:04.296114+0000","last_peered":"2024-04-26T22:28:04.296114+0000","last_clean":"2024-04-26T22:28:04.296114+0000","last_became_active":"2024-04-26T22:28:02.561843+0000","last_became_peered":"2024-04-26T22:28:02.561843+0000","last_unstale":"2024-04-26T22:28:04.296114+0000","last_undegraded":"2024-04-26T22:28:04.296114+0000","last_fullsized":"2024-04-26T22:28:04.296114+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141972+0000","last_change":"2024-04-26T22:28:02.556548+0000","last_active":"2024-04-26T22:28:04.141972+0000","last_peered":"2024-04-26T22:28:04.141972+0000","last_clean":"2024-04-26T22:28:04.141972+0000","last_became_active":"2024-04-26T22:28:02.556221+0000","last_became_peered":"2024-04-26T22:28:02.556221+0000","last_unstale":"2024-04-26T22:28:04.141972+0000","last_undegraded":"2024-04-26T22:28:04.141972+0000","last_fullsized":"2024-04-26T22:28:04.141972+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:07.397727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141697+0000","last_change":"2024-04-26T22:28:02.564065+0000","last_active":"2024-04-26T22:28:04.141697+0000","last_peered":"2024-04-26T22:28:04.141697+0000","last_clean":"2024-04-26T22:28:04.141697+0000","last_became_active":"2024-04-26T22:28:02.563390+0000","last_became_peered":"2024-04-26T22:28:02.563390+0000","last_unstale":"2024-04-26T22:28:04.141697+0000","last_undegraded":"2024-04-26T22:28:04.141697+0000","last_fullsized":"2024-04-26T22:28:04.141697+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560040+0000","last_change":"2024-04-26T22:28:02.569886+0000","last_active":"2024-04-26T22:28:03.560040+0000","last_peered":"2024-04-26T22:28:03.560040+0000","last_clean":"2024-04-26T22:28:03.560040+0000","last_became_active":"2024-04-26T22:28:02.569468+0000","last_became_peered":"2024-04-26T22:28:02.569468+0000","last_unstale":"2024-04-26T22:28:03.560040+0000","last_undegraded":"2024-04-26T22:28:03.560040+0000","last_fullsized":"2024-04-26T22:28:03.560040+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559451+0000","last_change":"2024-04-26T22:28:02.574061+0000","last_active":"2024-04-26T22:28:03.559451+0000","last_peered":"2024-04-26T22:28:03.559451+0000","last_clean":"2024-04-26T22:28:03.559451+0000","last_became_active":"2024-04-26T22:28:02.573739+0000","last_became_peered":"2024-04-26T22:28:02.573739+0000","last_unstale":"2024-04-26T22:28:03.559451+0000","last_undegraded":"2024-04-26T22:28:03.559451+0000","last_fullsized":"2024-04-26T22:28:03.559451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141961+0000","last_change":"2024-04-26T22:28:02.570383+0000","last_active":"2024-04-26T22:28:04.141961+0000","last_peered":"2024-04-26T22:28:04.141961+0000","last_clean":"2024-04-26T22:28:04.141961+0000","last_became_active":"2024-04-26T22:28:02.570047+0000","last_became_peered":"2024-04-26T22:28:02.570047+0000","last_unstale":"2024-04-26T22:28:04.141961+0000","last_undegraded":"2024-04-26T22:28:04.141961+0000","last_fullsized":"2024-04-26T22:28:04.141961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560841+0000","last_change":"2024-04-26T22:28:02.572697+0000","last_active":"2024-04-26T22:28:03.560841+0000","last_peered":"2024-04-26T22:28:03.560841+0000","last_clean":"2024-04-26T22:28:03.560841+0000","last_became_active":"2024-04-26T22:28:02.572395+0000","last_became_peered":"2024-04-26T22:28:02.572395+0000","last_unstale":"2024-04-26T22:28:03.560841+0000","last_undegraded":"2024-04-26T22:28:03.560841+0000","last_fullsized":"2024-04-26T22:28:03.560841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914855+0000","last_change":"2024-04-26T22:28:02.559102+0000","last_active":"2024-04-26T22:28:03.914855+0000","last_peered":"2024-04-26T22:28:03.914855+0000","last_clean":"2024-04-26T22:28:03.914855+0000","last_became_active":"2024-04-26T22:28:02.558834+0000","last_became_peered":"2024-04-26T22:28:02.558834+0000","last_unstale":"2024-04-26T22:28:03.914855+0000","last_undegraded":"2024-04-26T22:28:03.914855+0000","last_fullsized":"2024-04-26T22:28:03.914855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:44.309166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296202+0000","last_change":"2024-04-26T22:28:02.563547+0000","last_active":"2024-04-26T22:28:04.296202+0000","last_peered":"2024-04-26T22:28:04.296202+0000","last_clean":"2024-04-26T22:28:04.296202+0000","last_became_active":"2024-04-26T22:28:02.563212+0000","last_became_peered":"2024-04-26T22:28:02.563212+0000","last_unstale":"2024-04-26T22:28:04.296202+0000","last_undegraded":"2024-04-26T22:28:04.296202+0000","last_fullsized":"2024-04-26T22:28:04.296202+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.992166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560537+0000","last_change":"2024-04-26T22:28:02.554993+0000","last_active":"2024-04-26T22:28:03.560537+0000","last_peered":"2024-04-26T22:28:03.560537+0000","last_clean":"2024-04-26T22:28:03.560537+0000","last_became_active":"2024-04-26T22:28:02.554174+0000","last_became_peered":"2024-04-26T22:28:02.554174+0000","last_unstale":"2024-04-26T22:28:03.560537+0000","last_undegraded":"2024-04-26T22:28:03.560537+0000","last_fullsized":"2024-04-26T22:28:03.560537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142255+0000","last_change":"2024-04-26T22:28:02.565230+0000","last_active":"2024-04-26T22:28:04.142255+0000","last_peered":"2024-04-26T22:28:04.142255+0000","last_clean":"2024-04-26T22:28:04.142255+0000","last_became_active":"2024-04-26T22:28:02.564938+0000","last_became_peered":"2024-04-26T22:28:02.564938+0000","last_unstale":"2024-04-26T22:28:04.142255+0000","last_undegraded":"2024-04-26T22:28:04.142255+0000","last_fullsized":"2024-04-26T22:28:04.142255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142319+0000","last_change":"2024-04-26T22:28:02.557007+0000","last_active":"2024-04-26T22:28:04.142319+0000","last_peered":"2024-04-26T22:28:04.142319+0000","last_clean":"2024-04-26T22:28:04.142319+0000","last_became_active":"2024-04-26T22:28:02.556160+0000","last_became_peered":"2024-04-26T22:28:02.556160+0000","last_unstale":"2024-04-26T22:28:04.142319+0000","last_undegraded":"2024-04-26T22:28:04.142319+0000","last_fullsized":"2024-04-26T22:28:04.142319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.992166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296662+0000","last_change":"2024-04-26T22:28:02.571421+0000","last_active":"2024-04-26T22:28:04.296662+0000","last_peered":"2024-04-26T22:28:04.296662+0000","last_clean":"2024-04-26T22:28:04.296662+0000","last_became_active":"2024-04-26T22:28:02.571057+0000","last_became_peered":"2024-04-26T22:28:02.571057+0000","last_unstale":"2024-04-26T22:28:04.296662+0000","last_undegraded":"2024-04-26T22:28:04.296662+0000","last_fullsized":"2024-04-26T22:28:04.296662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.270062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296381+0000","last_change":"2024-04-26T22:28:02.572507+0000","last_active":"2024-04-26T22:28:04.296381+0000","last_peered":"2024-04-26T22:28:04.296381+0000","last_clean":"2024-04-26T22:28:04.296381+0000","last_became_active":"2024-04-26T22:28:02.572194+0000","last_became_peered":"2024-04-26T22:28:02.572194+0000","last_unstale":"2024-04-26T22:28:04.296381+0000","last_undegraded":"2024-04-26T22:28:04.296381+0000","last_fullsized":"2024-04-26T22:28:04.296381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:05.320316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915010+0000","last_change":"2024-04-26T22:28:02.556476+0000","last_active":"2024-04-26T22:28:03.915010+0000","last_peered":"2024-04-26T22:28:03.915010+0000","last_clean":"2024-04-26T22:28:03.915010+0000","last_became_active":"2024-04-26T22:28:02.556191+0000","last_became_peered":"2024-04-26T22:28:02.556191+0000","last_unstale":"2024-04-26T22:28:03.915010+0000","last_undegraded":"2024-04-26T22:28:03.915010+0000","last_fullsized":"2024-04-26T22:28:03.915010+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:08.189558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561210+0000","last_change":"2024-04-26T22:28:02.552028+0000","last_active":"2024-04-26T22:28:03.561210+0000","last_peered":"2024-04-26T22:28:03.561210+0000","last_clean":"2024-04-26T22:28:03.561210+0000","last_became_active":"2024-04-26T22:28:02.551742+0000","last_became_peered":"2024-04-26T22:28:02.551742+0000","last_unstale":"2024-04-26T22:28:03.561210+0000","last_undegraded":"2024-04-26T22:28:03.561210+0000","last_fullsized":"2024-04-26T22:28:03.561210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560233+0000","last_change":"2024-04-26T22:28:02.562010+0000","last_active":"2024-04-26T22:28:03.560233+0000","last_peered":"2024-04-26T22:28:03.560233+0000","last_clean":"2024-04-26T22:28:03.560233+0000","last_became_active":"2024-04-26T22:28:02.561719+0000","last_became_peered":"2024-04-26T22:28:02.561719+0000","last_unstale":"2024-04-26T22:28:03.560233+0000","last_undegraded":"2024-04-26T22:28:03.560233+0000","last_fullsized":"2024-04-26T22:28:03.560233+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554272+0000","last_change":"2024-04-26T22:28:02.554165+0000","last_active":"2024-04-26T22:28:03.554272+0000","last_peered":"2024-04-26T22:28:03.554272+0000","last_clean":"2024-04-26T22:28:03.554272+0000","last_became_active":"2024-04-26T22:28:02.553872+0000","last_became_peered":"2024-04-26T22:28:02.553872+0000","last_unstale":"2024-04-26T22:28:03.554272+0000","last_undegraded":"2024-04-26T22:28:03.554272+0000","last_fullsized":"2024-04-26T22:28:03.554272+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559588+0000","last_change":"2024-04-26T22:28:02.558912+0000","last_active":"2024-04-26T22:28:03.559588+0000","last_peered":"2024-04-26T22:28:03.559588+0000","last_clean":"2024-04-26T22:28:03.559588+0000","last_became_active":"2024-04-26T22:28:02.558633+0000","last_became_peered":"2024-04-26T22:28:02.558633+0000","last_unstale":"2024-04-26T22:28:03.559588+0000","last_undegraded":"2024-04-26T22:28:03.559588+0000","last_fullsized":"2024-04-26T22:28:03.559588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559564+0000","last_change":"2024-04-26T22:28:02.573937+0000","last_active":"2024-04-26T22:28:03.559564+0000","last_peered":"2024-04-26T22:28:03.559564+0000","last_clean":"2024-04-26T22:28:03.559564+0000","last_became_active":"2024-04-26T22:28:02.573553+0000","last_became_peered":"2024-04-26T22:28:02.573553+0000","last_unstale":"2024-04-26T22:28:03.559564+0000","last_undegraded":"2024-04-26T22:28:03.559564+0000","last_fullsized":"2024-04-26T22:28:03.559564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296785+0000","last_change":"2024-04-26T22:28:02.565075+0000","last_active":"2024-04-26T22:28:04.296785+0000","last_peered":"2024-04-26T22:28:04.296785+0000","last_clean":"2024-04-26T22:28:04.296785+0000","last_became_active":"2024-04-26T22:28:02.564553+0000","last_became_peered":"2024-04-26T22:28:02.564553+0000","last_unstale":"2024-04-26T22:28:04.296785+0000","last_undegraded":"2024-04-26T22:28:04.296785+0000","last_fullsized":"2024-04-26T22:28:04.296785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:11.425121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559403+0000","last_change":"2024-04-26T22:28:02.568815+0000","last_active":"2024-04-26T22:28:03.559403+0000","last_peered":"2024-04-26T22:28:03.559403+0000","last_clean":"2024-04-26T22:28:03.559403+0000","last_became_active":"2024-04-26T22:28:02.568165+0000","last_became_peered":"2024-04-26T22:28:02.568165+0000","last_unstale":"2024-04-26T22:28:03.559403+0000","last_undegraded":"2024-04-26T22:28:03.559403+0000","last_fullsized":"2024-04-26T22:28:03.559403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561227+0000","last_change":"2024-04-26T22:28:02.544685+0000","last_active":"2024-04-26T22:28:03.561227+0000","last_peered":"2024-04-26T22:28:03.561227+0000","last_clean":"2024-04-26T22:28:03.561227+0000","last_became_active":"2024-04-26T22:28:02.544398+0000","last_became_peered":"2024-04-26T22:28:02.544398+0000","last_unstale":"2024-04-26T22:28:03.561227+0000","last_undegraded":"2024-04-26T22:28:03.561227+0000","last_fullsized":"2024-04-26T22:28:03.561227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561298+0000","last_change":"2024-04-26T22:28:02.565191+0000","last_active":"2024-04-26T22:28:03.561298+0000","last_peered":"2024-04-26T22:28:03.561298+0000","last_clean":"2024-04-26T22:28:03.561298+0000","last_became_active":"2024-04-26T22:28:02.564949+0000","last_became_peered":"2024-04-26T22:28:02.564949+0000","last_unstale":"2024-04-26T22:28:03.561298+0000","last_undegraded":"2024-04-26T22:28:03.561298+0000","last_fullsized":"2024-04-26T22:28:03.561298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141913+0000","last_change":"2024-04-26T22:28:02.555628+0000","last_active":"2024-04-26T22:28:04.141913+0000","last_peered":"2024-04-26T22:28:04.141913+0000","last_clean":"2024-04-26T22:28:04.141913+0000","last_became_active":"2024-04-26T22:28:02.555353+0000","last_became_peered":"2024-04-26T22:28:02.555353+0000","last_unstale":"2024-04-26T22:28:04.141913+0000","last_undegraded":"2024-04-26T22:28:04.141913+0000","last_fullsized":"2024-04-26T22:28:04.141913+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093106+0000","last_change":"2024-04-26T22:28:02.564462+0000","last_active":"2024-04-26T22:28:04.093106+0000","last_peered":"2024-04-26T22:28:04.093106+0000","last_clean":"2024-04-26T22:28:04.093106+0000","last_became_active":"2024-04-26T22:28:02.563780+0000","last_became_peered":"2024-04-26T22:28:02.563780+0000","last_unstale":"2024-04-26T22:28:04.093106+0000","last_undegraded":"2024-04-26T22:28:04.093106+0000","last_fullsized":"2024-04-26T22:28:04.093106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.595312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141684+0000","last_change":"2024-04-26T22:28:02.560240+0000","last_active":"2024-04-26T22:28:04.141684+0000","last_peered":"2024-04-26T22:28:04.141684+0000","last_clean":"2024-04-26T22:28:04.141684+0000","last_became_active":"2024-04-26T22:28:02.559907+0000","last_became_peered":"2024-04-26T22:28:02.559907+0000","last_unstale":"2024-04-26T22:28:04.141684+0000","last_undegraded":"2024-04-26T22:28:04.141684+0000","last_fullsized":"2024-04-26T22:28:04.141684+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554421+0000","last_change":"2024-04-26T22:28:02.554463+0000","last_active":"2024-04-26T22:28:03.554421+0000","last_peered":"2024-04-26T22:28:03.554421+0000","last_clean":"2024-04-26T22:28:03.554421+0000","last_became_active":"2024-04-26T22:28:02.553837+0000","last_became_peered":"2024-04-26T22:28:02.553837+0000","last_unstale":"2024-04-26T22:28:03.554421+0000","last_undegraded":"2024-04-26T22:28:03.554421+0000","last_fullsized":"2024-04-26T22:28:03.554421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093407+0000","last_change":"2024-04-26T22:28:01.551148+0000","last_active":"2024-04-26T22:28:04.093407+0000","last_peered":"2024-04-26T22:28:04.093407+0000","last_clean":"2024-04-26T22:28:04.093407+0000","last_became_active":"2024-04-26T22:28:01.550854+0000","last_became_peered":"2024-04-26T22:28:01.550854+0000","last_unstale":"2024-04-26T22:28:04.093407+0000","last_undegraded":"2024-04-26T22:28:04.093407+0000","last_fullsized":"2024-04-26T22:28:04.093407+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093370+0000","last_change":"2024-04-26T22:28:02.553397+0000","last_active":"2024-04-26T22:28:04.093370+0000","last_peered":"2024-04-26T22:28:04.093370+0000","last_clean":"2024-04-26T22:28:04.093370+0000","last_became_active":"2024-04-26T22:28:02.553087+0000","last_became_peered":"2024-04-26T22:28:02.553087+0000","last_unstale":"2024-04-26T22:28:04.093370+0000","last_undegraded":"2024-04-26T22:28:04.093370+0000","last_fullsized":"2024-04-26T22:28:04.093370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554403+0000","last_change":"2024-04-26T22:28:01.553286+0000","last_active":"2024-04-26T22:28:03.554403+0000","last_peered":"2024-04-26T22:28:03.554403+0000","last_clean":"2024-04-26T22:28:03.554403+0000","last_became_active":"2024-04-26T22:28:01.552926+0000","last_became_peered":"2024-04-26T22:28:01.552926+0000","last_unstale":"2024-04-26T22:28:03.554403+0000","last_undegraded":"2024-04-26T22:28:03.554403+0000","last_fullsized":"2024-04-26T22:28:03.554403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560171+0000","last_change":"2024-04-26T22:28:02.569421+0000","last_active":"2024-04-26T22:28:03.560171+0000","last_peered":"2024-04-26T22:28:03.560171+0000","last_clean":"2024-04-26T22:28:03.560171+0000","last_became_active":"2024-04-26T22:28:02.569122+0000","last_became_peered":"2024-04-26T22:28:02.569122+0000","last_unstale":"2024-04-26T22:28:03.560171+0000","last_undegraded":"2024-04-26T22:28:03.560171+0000","last_fullsized":"2024-04-26T22:28:03.560171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:55.085718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296345+0000","last_change":"2024-04-26T22:28:01.559844+0000","last_active":"2024-04-26T22:28:04.296345+0000","last_peered":"2024-04-26T22:28:04.296345+0000","last_clean":"2024-04-26T22:28:04.296345+0000","last_became_active":"2024-04-26T22:28:01.559546+0000","last_became_peered":"2024-04-26T22:28:01.559546+0000","last_unstale":"2024-04-26T22:28:04.296345+0000","last_undegraded":"2024-04-26T22:28:04.296345+0000","last_fullsized":"2024-04-26T22:28:04.296345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559452+0000","last_change":"2024-04-26T22:28:02.568495+0000","last_active":"2024-04-26T22:28:03.559452+0000","last_peered":"2024-04-26T22:28:03.559452+0000","last_clean":"2024-04-26T22:28:03.559452+0000","last_became_active":"2024-04-26T22:28:02.567595+0000","last_became_peered":"2024-04-26T22:28:02.567595+0000","last_unstale":"2024-04-26T22:28:03.559452+0000","last_undegraded":"2024-04-26T22:28:03.559452+0000","last_fullsized":"2024-04-26T22:28:03.559452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915031+0000","last_change":"2024-04-26T22:28:01.552326+0000","last_active":"2024-04-26T22:28:03.915031+0000","last_peered":"2024-04-26T22:28:03.915031+0000","last_clean":"2024-04-26T22:28:03.915031+0000","last_became_active":"2024-04-26T22:28:01.551423+0000","last_became_peered":"2024-04-26T22:28:01.551423+0000","last_unstale":"2024-04-26T22:28:03.915031+0000","last_undegraded":"2024-04-26T22:28:03.915031+0000","last_fullsized":"2024-04-26T22:28:03.915031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:41.694091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142012+0000","last_change":"2024-04-26T22:28:02.560673+0000","last_active":"2024-04-26T22:28:04.142012+0000","last_peered":"2024-04-26T22:28:04.142012+0000","last_clean":"2024-04-26T22:28:04.142012+0000","last_became_active":"2024-04-26T22:28:02.560395+0000","last_became_peered":"2024-04-26T22:28:02.560395+0000","last_unstale":"2024-04-26T22:28:04.142012+0000","last_undegraded":"2024-04-26T22:28:04.142012+0000","last_fullsized":"2024-04-26T22:28:04.142012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561334+0000","last_change":"2024-04-26T22:28:01.549390+0000","last_active":"2024-04-26T22:28:03.561334+0000","last_peered":"2024-04-26T22:28:03.561334+0000","last_clean":"2024-04-26T22:28:03.561334+0000","last_became_active":"2024-04-26T22:28:01.548063+0000","last_became_peered":"2024-04-26T22:28:01.548063+0000","last_unstale":"2024-04-26T22:28:03.561334+0000","last_undegraded":"2024-04-26T22:28:03.561334+0000","last_fullsized":"2024-04-26T22:28:03.561334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560519+0000","last_change":"2024-04-26T22:28:02.544922+0000","last_active":"2024-04-26T22:28:03.560519+0000","last_peered":"2024-04-26T22:28:03.560519+0000","last_clean":"2024-04-26T22:28:03.560519+0000","last_became_active":"2024-04-26T22:28:02.544326+0000","last_became_peered":"2024-04-26T22:28:02.544326+0000","last_unstale":"2024-04-26T22:28:03.560519+0000","last_undegraded":"2024-04-26T22:28:03.560519+0000","last_fullsized":"2024-04-26T22:28:03.560519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560139+0000","last_change":"2024-04-26T22:28:01.548404+0000","last_active":"2024-04-26T22:28:03.560139+0000","last_peered":"2024-04-26T22:28:03.560139+0000","last_clean":"2024-04-26T22:28:03.560139+0000","last_became_active":"2024-04-26T22:28:01.548088+0000","last_became_peered":"2024-04-26T22:28:01.548088+0000","last_unstale":"2024-04-26T22:28:03.560139+0000","last_undegraded":"2024-04-26T22:28:03.560139+0000","last_fullsized":"2024-04-26T22:28:03.560139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559971+0000","last_change":"2024-04-26T22:28:02.554967+0000","last_active":"2024-04-26T22:28:03.559971+0000","last_peered":"2024-04-26T22:28:03.559971+0000","last_clean":"2024-04-26T22:28:03.559971+0000","last_became_active":"2024-04-26T22:28:02.554198+0000","last_became_peered":"2024-04-26T22:28:02.554198+0000","last_unstale":"2024-04-26T22:28:03.559971+0000","last_undegraded":"2024-04-26T22:28:03.559971+0000","last_fullsized":"2024-04-26T22:28:03.559971+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296024+0000","last_change":"2024-04-26T22:28:01.556939+0000","last_active":"2024-04-26T22:28:04.296024+0000","last_peered":"2024-04-26T22:28:04.296024+0000","last_clean":"2024-04-26T22:28:04.296024+0000","last_became_active":"2024-04-26T22:28:01.556663+0000","last_became_peered":"2024-04-26T22:28:01.556663+0000","last_unstale":"2024-04-26T22:28:04.296024+0000","last_undegraded":"2024-04-26T22:28:04.296024+0000","last_fullsized":"2024-04-26T22:28:04.296024+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:23.588575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559050+0000","last_change":"2024-04-26T22:28:02.561770+0000","last_active":"2024-04-26T22:28:03.559050+0000","last_peered":"2024-04-26T22:28:03.559050+0000","last_clean":"2024-04-26T22:28:03.559050+0000","last_became_active":"2024-04-26T22:28:02.561463+0000","last_became_peered":"2024-04-26T22:28:02.561463+0000","last_unstale":"2024-04-26T22:28:03.559050+0000","last_undegraded":"2024-04-26T22:28:03.559050+0000","last_fullsized":"2024-04-26T22:28:03.559050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:55.085718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559920+0000","last_change":"2024-04-26T22:28:01.552353+0000","last_active":"2024-04-26T22:28:03.559920+0000","last_peered":"2024-04-26T22:28:03.559920+0000","last_clean":"2024-04-26T22:28:03.559920+0000","last_became_active":"2024-04-26T22:28:01.549925+0000","last_became_peered":"2024-04-26T22:28:01.549925+0000","last_unstale":"2024-04-26T22:28:03.559920+0000","last_undegraded":"2024-04-26T22:28:03.559920+0000","last_fullsized":"2024-04-26T22:28:03.559920+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:47.117169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560778+0000","last_change":"2024-04-26T22:28:02.566415+0000","last_active":"2024-04-26T22:28:03.560778+0000","last_peered":"2024-04-26T22:28:03.560778+0000","last_clean":"2024-04-26T22:28:03.560778+0000","last_became_active":"2024-04-26T22:28:02.566124+0000","last_became_peered":"2024-04-26T22:28:02.566124+0000","last_unstale":"2024-04-26T22:28:03.560778+0000","last_undegraded":"2024-04-26T22:28:03.560778+0000","last_fullsized":"2024-04-26T22:28:03.560778+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296623+0000","last_change":"2024-04-26T22:28:01.555381+0000","last_active":"2024-04-26T22:28:04.296623+0000","last_peered":"2024-04-26T22:28:04.296623+0000","last_clean":"2024-04-26T22:28:04.296623+0000","last_became_active":"2024-04-26T22:28:01.555132+0000","last_became_peered":"2024-04-26T22:28:01.555132+0000","last_unstale":"2024-04-26T22:28:04.296623+0000","last_undegraded":"2024-04-26T22:28:04.296623+0000","last_fullsized":"2024-04-26T22:28:04.296623+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:40.586767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296660+0000","last_change":"2024-04-26T22:28:02.572112+0000","last_active":"2024-04-26T22:28:04.296660+0000","last_peered":"2024-04-26T22:28:04.296660+0000","last_clean":"2024-04-26T22:28:04.296660+0000","last_became_active":"2024-04-26T22:28:02.571821+0000","last_became_peered":"2024-04-26T22:28:02.571821+0000","last_unstale":"2024-04-26T22:28:04.296660+0000","last_undegraded":"2024-04-26T22:28:04.296660+0000","last_fullsized":"2024-04-26T22:28:04.296660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:08.426549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554484+0000","last_change":"2024-04-26T22:28:01.563856+0000","last_active":"2024-04-26T22:28:03.554484+0000","last_peered":"2024-04-26T22:28:03.554484+0000","last_clean":"2024-04-26T22:28:03.554484+0000","last_became_active":"2024-04-26T22:28:01.563514+0000","last_became_peered":"2024-04-26T22:28:01.563514+0000","last_unstale":"2024-04-26T22:28:03.554484+0000","last_undegraded":"2024-04-26T22:28:03.554484+0000","last_fullsized":"2024-04-26T22:28:03.554484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296371+0000","last_change":"2024-04-26T22:28:02.566484+0000","last_active":"2024-04-26T22:28:04.296371+0000","last_peered":"2024-04-26T22:28:04.296371+0000","last_clean":"2024-04-26T22:28:04.296371+0000","last_became_active":"2024-04-26T22:28:02.565731+0000","last_became_peered":"2024-04-26T22:28:02.565731+0000","last_unstale":"2024-04-26T22:28:04.296371+0000","last_undegraded":"2024-04-26T22:28:04.296371+0000","last_fullsized":"2024-04-26T22:28:04.296371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:43.011198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561005+0000","last_change":"2024-04-26T22:28:01.553958+0000","last_active":"2024-04-26T22:28:03.561005+0000","last_peered":"2024-04-26T22:28:03.561005+0000","last_clean":"2024-04-26T22:28:03.561005+0000","last_became_active":"2024-04-26T22:28:01.553552+0000","last_became_peered":"2024-04-26T22:28:01.553552+0000","last_unstale":"2024-04-26T22:28:03.561005+0000","last_undegraded":"2024-04-26T22:28:03.561005+0000","last_fullsized":"2024-04-26T22:28:03.561005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560821+0000","last_change":"2024-04-26T22:28:02.568051+0000","last_active":"2024-04-26T22:28:03.560821+0000","last_peered":"2024-04-26T22:28:03.560821+0000","last_clean":"2024-04-26T22:28:03.560821+0000","last_became_active":"2024-04-26T22:28:02.567797+0000","last_became_peered":"2024-04-26T22:28:02.567797+0000","last_unstale":"2024-04-26T22:28:03.560821+0000","last_undegraded":"2024-04-26T22:28:03.560821+0000","last_fullsized":"2024-04-26T22:28:03.560821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296616+0000","last_change":"2024-04-26T22:28:01.559144+0000","last_active":"2024-04-26T22:28:04.296616+0000","last_peered":"2024-04-26T22:28:04.296616+0000","last_clean":"2024-04-26T22:28:04.296616+0000","last_became_active":"2024-04-26T22:28:01.558496+0000","last_became_peered":"2024-04-26T22:28:01.558496+0000","last_unstale":"2024-04-26T22:28:04.296616+0000","last_undegraded":"2024-04-26T22:28:04.296616+0000","last_fullsized":"2024-04-26T22:28:04.296616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559095+0000","last_change":"2024-04-26T22:28:02.567564+0000","last_active":"2024-04-26T22:28:03.559095+0000","last_peered":"2024-04-26T22:28:03.559095+0000","last_clean":"2024-04-26T22:28:03.559095+0000","last_became_active":"2024-04-26T22:28:02.567232+0000","last_became_peered":"2024-04-26T22:28:02.567232+0000","last_unstale":"2024-04-26T22:28:03.559095+0000","last_undegraded":"2024-04-26T22:28:03.559095+0000","last_fullsized":"2024-04-26T22:28:03.559095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561409+0000","last_change":"2024-04-26T22:28:01.550978+0000","last_active":"2024-04-26T22:28:03.561409+0000","last_peered":"2024-04-26T22:28:03.561409+0000","last_clean":"2024-04-26T22:28:03.561409+0000","last_became_active":"2024-04-26T22:28:01.550154+0000","last_became_peered":"2024-04-26T22:28:01.550154+0000","last_unstale":"2024-04-26T22:28:03.561409+0000","last_undegraded":"2024-04-26T22:28:03.561409+0000","last_fullsized":"2024-04-26T22:28:03.561409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:09.972664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296276+0000","last_change":"2024-04-26T22:28:02.562944+0000","last_active":"2024-04-26T22:28:04.296276+0000","last_peered":"2024-04-26T22:28:04.296276+0000","last_clean":"2024-04-26T22:28:04.296276+0000","last_became_active":"2024-04-26T22:28:02.562580+0000","last_became_peered":"2024-04-26T22:28:02.562580+0000","last_unstale":"2024-04-26T22:28:04.296276+0000","last_undegraded":"2024-04-26T22:28:04.296276+0000","last_fullsized":"2024-04-26T22:28:04.296276+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:32.066791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559152+0000","last_change":"2024-04-26T22:28:01.551899+0000","last_active":"2024-04-26T22:28:03.559152+0000","last_peered":"2024-04-26T22:28:03.559152+0000","last_clean":"2024-04-26T22:28:03.559152+0000","last_became_active":"2024-04-26T22:28:01.551587+0000","last_became_peered":"2024-04-26T22:28:01.551587+0000","last_unstale":"2024-04-26T22:28:03.559152+0000","last_undegraded":"2024-04-26T22:28:03.559152+0000","last_fullsized":"2024-04-26T22:28:03.559152+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142081+0000","last_change":"2024-04-26T22:28:02.560842+0000","last_active":"2024-04-26T22:28:04.142081+0000","last_peered":"2024-04-26T22:28:04.142081+0000","last_clean":"2024-04-26T22:28:04.142081+0000","last_became_active":"2024-04-26T22:28:02.560572+0000","last_became_peered":"2024-04-26T22:28:02.560572+0000","last_unstale":"2024-04-26T22:28:04.142081+0000","last_undegraded":"2024-04-26T22:28:04.142081+0000","last_fullsized":"2024-04-26T22:28:04.142081+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296115+0000","last_change":"2024-04-26T22:28:01.555910+0000","last_active":"2024-04-26T22:28:04.296115+0000","last_peered":"2024-04-26T22:28:04.296115+0000","last_clean":"2024-04-26T22:28:04.296115+0000","last_became_active":"2024-04-26T22:28:01.555596+0000","last_became_peered":"2024-04-26T22:28:01.555596+0000","last_unstale":"2024-04-26T22:28:04.296115+0000","last_undegraded":"2024-04-26T22:28:04.296115+0000","last_fullsized":"2024-04-26T22:28:04.296115+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:35.686888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093195+0000","last_change":"2024-04-26T22:28:02.559888+0000","last_active":"2024-04-26T22:28:04.093195+0000","last_peered":"2024-04-26T22:28:04.093195+0000","last_clean":"2024-04-26T22:28:04.093195+0000","last_became_active":"2024-04-26T22:28:02.559626+0000","last_became_peered":"2024-04-26T22:28:02.559626+0000","last_unstale":"2024-04-26T22:28:04.093195+0000","last_undegraded":"2024-04-26T22:28:04.093195+0000","last_fullsized":"2024-04-26T22:28:04.093195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296087+0000","last_change":"2024-04-26T22:28:01.559695+0000","last_active":"2024-04-26T22:28:04.296087+0000","last_peered":"2024-04-26T22:28:04.296087+0000","last_clean":"2024-04-26T22:28:04.296087+0000","last_became_active":"2024-04-26T22:28:01.559448+0000","last_became_peered":"2024-04-26T22:28:01.559448+0000","last_unstale":"2024-04-26T22:28:04.296087+0000","last_undegraded":"2024-04-26T22:28:04.296087+0000","last_fullsized":"2024-04-26T22:28:04.296087+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:17.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559868+0000","last_change":"2024-04-26T22:28:02.553976+0000","last_active":"2024-04-26T22:28:03.559868+0000","last_peered":"2024-04-26T22:28:03.559868+0000","last_clean":"2024-04-26T22:28:03.559868+0000","last_became_active":"2024-04-26T22:28:02.553134+0000","last_became_peered":"2024-04-26T22:28:02.553134+0000","last_unstale":"2024-04-26T22:28:03.559868+0000","last_undegraded":"2024-04-26T22:28:03.559868+0000","last_fullsized":"2024-04-26T22:28:03.559868+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559673+0000","last_change":"2024-04-26T22:28:01.560134+0000","last_active":"2024-04-26T22:28:03.559673+0000","last_peered":"2024-04-26T22:28:03.559673+0000","last_clean":"2024-04-26T22:28:03.559673+0000","last_became_active":"2024-04-26T22:28:01.559832+0000","last_became_peered":"2024-04-26T22:28:01.559832+0000","last_unstale":"2024-04-26T22:28:03.559673+0000","last_undegraded":"2024-04-26T22:28:03.559673+0000","last_fullsized":"2024-04-26T22:28:03.559673+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561020+0000","last_change":"2024-04-26T22:28:02.549060+0000","last_active":"2024-04-26T22:28:03.561020+0000","last_peered":"2024-04-26T22:28:03.561020+0000","last_clean":"2024-04-26T22:28:03.561020+0000","last_became_active":"2024-04-26T22:28:02.548805+0000","last_became_peered":"2024-04-26T22:28:02.548805+0000","last_unstale":"2024-04-26T22:28:03.561020+0000","last_undegraded":"2024-04-26T22:28:03.561020+0000","last_fullsized":"2024-04-26T22:28:03.561020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559988+0000","last_change":"2024-04-26T22:28:01.550770+0000","last_active":"2024-04-26T22:28:03.559988+0000","last_peered":"2024-04-26T22:28:03.559988+0000","last_clean":"2024-04-26T22:28:03.559988+0000","last_became_active":"2024-04-26T22:28:01.550490+0000","last_became_peered":"2024-04-26T22:28:01.550490+0000","last_unstale":"2024-04-26T22:28:03.559988+0000","last_undegraded":"2024-04-26T22:28:03.559988+0000","last_fullsized":"2024-04-26T22:28:03.559988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:43.302429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559225+0000","last_change":"2024-04-26T22:28:02.562897+0000","last_active":"2024-04-26T22:28:03.559225+0000","last_peered":"2024-04-26T22:28:03.559225+0000","last_clean":"2024-04-26T22:28:03.559225+0000","last_became_active":"2024-04-26T22:28:02.562115+0000","last_became_peered":"2024-04-26T22:28:02.562115+0000","last_unstale":"2024-04-26T22:28:03.559225+0000","last_undegraded":"2024-04-26T22:28:03.559225+0000","last_fullsized":"2024-04-26T22:28:03.559225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560102+0000","last_change":"2024-04-26T22:28:01.548123+0000","last_active":"2024-04-26T22:28:03.560102+0000","last_peered":"2024-04-26T22:28:03.560102+0000","last_clean":"2024-04-26T22:28:03.560102+0000","last_became_active":"2024-04-26T22:28:01.547847+0000","last_became_peered":"2024-04-26T22:28:01.547847+0000","last_unstale":"2024-04-26T22:28:03.560102+0000","last_undegraded":"2024-04-26T22:28:03.560102+0000","last_fullsized":"2024-04-26T22:28:03.560102+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915066+0000","last_change":"2024-04-26T22:28:02.556960+0000","last_active":"2024-04-26T22:28:03.915066+0000","last_peered":"2024-04-26T22:28:03.915066+0000","last_clean":"2024-04-26T22:28:03.915066+0000","last_became_active":"2024-04-26T22:28:02.556640+0000","last_became_peered":"2024-04-26T22:28:02.556640+0000","last_unstale":"2024-04-26T22:28:03.915066+0000","last_undegraded":"2024-04-26T22:28:03.915066+0000","last_fullsized":"2024-04-26T22:28:03.915066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:37.889245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560209+0000","last_change":"2024-04-26T22:28:01.548757+0000","last_active":"2024-04-26T22:28:03.560209+0000","last_peered":"2024-04-26T22:28:03.560209+0000","last_clean":"2024-04-26T22:28:03.560209+0000","last_became_active":"2024-04-26T22:28:01.548498+0000","last_became_peered":"2024-04-26T22:28:01.548498+0000","last_unstale":"2024-04-26T22:28:03.560209+0000","last_undegraded":"2024-04-26T22:28:03.560209+0000","last_fullsized":"2024-04-26T22:28:03.560209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:09.306834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914858+0000","last_change":"2024-04-26T22:28:02.560084+0000","last_active":"2024-04-26T22:28:03.914858+0000","last_peered":"2024-04-26T22:28:03.914858+0000","last_clean":"2024-04-26T22:28:03.914858+0000","last_became_active":"2024-04-26T22:28:02.559791+0000","last_became_peered":"2024-04-26T22:28:02.559791+0000","last_unstale":"2024-04-26T22:28:03.914858+0000","last_undegraded":"2024-04-26T22:28:03.914858+0000","last_fullsized":"2024-04-26T22:28:03.914858+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:48.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554539+0000","last_change":"2024-04-26T22:28:01.559890+0000","last_active":"2024-04-26T22:28:03.554539+0000","last_peered":"2024-04-26T22:28:03.554539+0000","last_clean":"2024-04-26T22:28:03.554539+0000","last_became_active":"2024-04-26T22:28:01.559564+0000","last_became_peered":"2024-04-26T22:28:01.559564+0000","last_unstale":"2024-04-26T22:28:03.554539+0000","last_undegraded":"2024-04-26T22:28:03.554539+0000","last_fullsized":"2024-04-26T22:28:03.554539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915035+0000","last_change":"2024-04-26T22:28:02.563648+0000","last_active":"2024-04-26T22:28:03.915035+0000","last_peered":"2024-04-26T22:28:03.915035+0000","last_clean":"2024-04-26T22:28:03.915035+0000","last_became_active":"2024-04-26T22:28:02.562244+0000","last_became_peered":"2024-04-26T22:28:02.562244+0000","last_unstale":"2024-04-26T22:28:03.915035+0000","last_undegraded":"2024-04-26T22:28:03.915035+0000","last_fullsized":"2024-04-26T22:28:03.915035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:12.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093248+0000","last_change":"2024-04-26T22:28:01.558972+0000","last_active":"2024-04-26T22:28:04.093248+0000","last_peered":"2024-04-26T22:28:04.093248+0000","last_clean":"2024-04-26T22:28:04.093248+0000","last_became_active":"2024-04-26T22:28:01.558693+0000","last_became_peered":"2024-04-26T22:28:01.558693+0000","last_unstale":"2024-04-26T22:28:04.093248+0000","last_undegraded":"2024-04-26T22:28:04.093248+0000","last_fullsized":"2024-04-26T22:28:04.093248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914721+0000","last_change":"2024-04-26T22:28:02.562354+0000","last_active":"2024-04-26T22:28:03.914721+0000","last_peered":"2024-04-26T22:28:03.914721+0000","last_clean":"2024-04-26T22:28:03.914721+0000","last_became_active":"2024-04-26T22:28:02.562094+0000","last_became_peered":"2024-04-26T22:28:02.562094+0000","last_unstale":"2024-04-26T22:28:03.914721+0000","last_undegraded":"2024-04-26T22:28:03.914721+0000","last_fullsized":"2024-04-26T22:28:03.914721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:10.313571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560258+0000","last_change":"2024-04-26T22:28:01.551281+0000","last_active":"2024-04-26T22:28:03.560258+0000","last_peered":"2024-04-26T22:28:03.560258+0000","last_clean":"2024-04-26T22:28:03.560258+0000","last_became_active":"2024-04-26T22:28:01.550195+0000","last_became_peered":"2024-04-26T22:28:01.550195+0000","last_unstale":"2024-04-26T22:28:03.560258+0000","last_undegraded":"2024-04-26T22:28:03.560258+0000","last_fullsized":"2024-04-26T22:28:03.560258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141785+0000","last_change":"2024-04-26T22:28:02.556199+0000","last_active":"2024-04-26T22:28:04.141785+0000","last_peered":"2024-04-26T22:28:04.141785+0000","last_clean":"2024-04-26T22:28:04.141785+0000","last_became_active":"2024-04-26T22:28:02.555904+0000","last_became_peered":"2024-04-26T22:28:02.555904+0000","last_unstale":"2024-04-26T22:28:04.141785+0000","last_undegraded":"2024-04-26T22:28:04.141785+0000","last_fullsized":"2024-04-26T22:28:04.141785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559469+0000","last_change":"2024-04-26T22:28:01.553831+0000","last_active":"2024-04-26T22:28:03.559469+0000","last_peered":"2024-04-26T22:28:03.559469+0000","last_clean":"2024-04-26T22:28:03.559469+0000","last_became_active":"2024-04-26T22:28:01.553563+0000","last_became_peered":"2024-04-26T22:28:01.553563+0000","last_unstale":"2024-04-26T22:28:03.559469+0000","last_undegraded":"2024-04-26T22:28:03.559469+0000","last_fullsized":"2024-04-26T22:28:03.559469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915107+0000","last_change":"2024-04-26T22:28:02.557112+0000","last_active":"2024-04-26T22:28:03.915107+0000","last_peered":"2024-04-26T22:28:03.915107+0000","last_clean":"2024-04-26T22:28:03.915107+0000","last_became_active":"2024-04-26T22:28:02.556819+0000","last_became_peered":"2024-04-26T22:28:02.556819+0000","last_unstale":"2024-04-26T22:28:03.915107+0000","last_undegraded":"2024-04-26T22:28:03.915107+0000","last_fullsized":"2024-04-26T22:28:03.915107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559687+0000","last_change":"2024-04-26T22:28:01.558893+0000","last_active":"2024-04-26T22:28:03.559687+0000","last_peered":"2024-04-26T22:28:03.559687+0000","last_clean":"2024-04-26T22:28:03.559687+0000","last_became_active":"2024-04-26T22:28:01.558541+0000","last_became_peered":"2024-04-26T22:28:01.558541+0000","last_unstale":"2024-04-26T22:28:03.559687+0000","last_undegraded":"2024-04-26T22:28:03.559687+0000","last_fullsized":"2024-04-26T22:28:03.559687+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561112+0000","last_change":"2024-04-26T22:28:02.569686+0000","last_active":"2024-04-26T22:28:03.561112+0000","last_peered":"2024-04-26T22:28:03.561112+0000","last_clean":"2024-04-26T22:28:03.561112+0000","last_became_active":"2024-04-26T22:28:02.568741+0000","last_became_peered":"2024-04-26T22:28:02.568741+0000","last_unstale":"2024-04-26T22:28:03.561112+0000","last_undegraded":"2024-04-26T22:28:03.561112+0000","last_fullsized":"2024-04-26T22:28:03.561112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554552+0000","last_change":"2024-04-26T22:28:01.560436+0000","last_active":"2024-04-26T22:28:03.554552+0000","last_peered":"2024-04-26T22:28:03.554552+0000","last_clean":"2024-04-26T22:28:03.554552+0000","last_became_active":"2024-04-26T22:28:01.560153+0000","last_became_peered":"2024-04-26T22:28:01.560153+0000","last_unstale":"2024-04-26T22:28:03.554552+0000","last_undegraded":"2024-04-26T22:28:03.554552+0000","last_fullsized":"2024-04-26T22:28:03.554552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:59.038151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554398+0000","last_change":"2024-04-26T22:28:02.569034+0000","last_active":"2024-04-26T22:28:03.554398+0000","last_peered":"2024-04-26T22:28:03.554398+0000","last_clean":"2024-04-26T22:28:03.554398+0000","last_became_active":"2024-04-26T22:28:02.568666+0000","last_became_peered":"2024-04-26T22:28:02.568666+0000","last_unstale":"2024-04-26T22:28:03.554398+0000","last_undegraded":"2024-04-26T22:28:03.554398+0000","last_fullsized":"2024-04-26T22:28:03.554398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559560+0000","last_change":"2024-04-26T22:28:01.558609+0000","last_active":"2024-04-26T22:28:03.559560+0000","last_peered":"2024-04-26T22:28:03.559560+0000","last_clean":"2024-04-26T22:28:03.559560+0000","last_became_active":"2024-04-26T22:28:01.558332+0000","last_became_peered":"2024-04-26T22:28:01.558332+0000","last_unstale":"2024-04-26T22:28:03.559560+0000","last_undegraded":"2024-04-26T22:28:03.559560+0000","last_fullsized":"2024-04-26T22:28:03.559560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093237+0000","last_change":"2024-04-26T22:28:02.559178+0000","last_active":"2024-04-26T22:28:04.093237+0000","last_peered":"2024-04-26T22:28:04.093237+0000","last_clean":"2024-04-26T22:28:04.093237+0000","last_became_active":"2024-04-26T22:28:02.558903+0000","last_became_peered":"2024-04-26T22:28:02.558903+0000","last_unstale":"2024-04-26T22:28:04.093237+0000","last_undegraded":"2024-04-26T22:28:04.093237+0000","last_fullsized":"2024-04-26T22:28:04.093237+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554646+0000","last_change":"2024-04-26T22:28:01.551370+0000","last_active":"2024-04-26T22:28:03.554646+0000","last_peered":"2024-04-26T22:28:03.554646+0000","last_clean":"2024-04-26T22:28:03.554646+0000","last_became_active":"2024-04-26T22:28:01.550977+0000","last_became_peered":"2024-04-26T22:28:01.550977+0000","last_unstale":"2024-04-26T22:28:03.554646+0000","last_undegraded":"2024-04-26T22:28:03.554646+0000","last_fullsized":"2024-04-26T22:28:03.554646+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554440+0000","last_change":"2024-04-26T22:28:02.555664+0000","last_active":"2024-04-26T22:28:03.554440+0000","last_peered":"2024-04-26T22:28:03.554440+0000","last_clean":"2024-04-26T22:28:03.554440+0000","last_became_active":"2024-04-26T22:28:02.555366+0000","last_became_peered":"2024-04-26T22:28:02.555366+0000","last_unstale":"2024-04-26T22:28:03.554440+0000","last_undegraded":"2024-04-26T22:28:03.554440+0000","last_fullsized":"2024-04-26T22:28:03.554440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559189+0000","last_change":"2024-04-26T22:28:01.555714+0000","last_active":"2024-04-26T22:28:03.559189+0000","last_peered":"2024-04-26T22:28:03.559189+0000","last_clean":"2024-04-26T22:28:03.559189+0000","last_became_active":"2024-04-26T22:28:01.555437+0000","last_became_peered":"2024-04-26T22:28:01.555437+0000","last_unstale":"2024-04-26T22:28:03.559189+0000","last_undegraded":"2024-04-26T22:28:03.559189+0000","last_fullsized":"2024-04-26T22:28:03.559189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296056+0000","last_change":"2024-04-26T22:28:02.570475+0000","last_active":"2024-04-26T22:28:04.296056+0000","last_peered":"2024-04-26T22:28:04.296056+0000","last_clean":"2024-04-26T22:28:04.296056+0000","last_became_active":"2024-04-26T22:28:02.570131+0000","last_became_peered":"2024-04-26T22:28:02.570131+0000","last_unstale":"2024-04-26T22:28:04.296056+0000","last_undegraded":"2024-04-26T22:28:04.296056+0000","last_fullsized":"2024-04-26T22:28:04.296056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560297+0000","last_change":"2024-04-26T22:28:01.549998+0000","last_active":"2024-04-26T22:28:03.560297+0000","last_peered":"2024-04-26T22:28:03.560297+0000","last_clean":"2024-04-26T22:28:03.560297+0000","last_became_active":"2024-04-26T22:28:01.549526+0000","last_became_peered":"2024-04-26T22:28:01.549526+0000","last_unstale":"2024-04-26T22:28:03.560297+0000","last_undegraded":"2024-04-26T22:28:03.560297+0000","last_fullsized":"2024-04-26T22:28:03.560297+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560047+0000","last_change":"2024-04-26T22:28:02.565124+0000","last_active":"2024-04-26T22:28:03.560047+0000","last_peered":"2024-04-26T22:28:03.560047+0000","last_clean":"2024-04-26T22:28:03.560047+0000","last_became_active":"2024-04-26T22:28:02.564865+0000","last_became_peered":"2024-04-26T22:28:02.564865+0000","last_unstale":"2024-04-26T22:28:03.560047+0000","last_undegraded":"2024-04-26T22:28:03.560047+0000","last_fullsized":"2024-04-26T22:28:03.560047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296172+0000","last_change":"2024-04-26T22:28:01.555358+0000","last_active":"2024-04-26T22:28:04.296172+0000","last_peered":"2024-04-26T22:28:04.296172+0000","last_clean":"2024-04-26T22:28:04.296172+0000","last_became_active":"2024-04-26T22:28:01.555056+0000","last_became_peered":"2024-04-26T22:28:01.555056+0000","last_unstale":"2024-04-26T22:28:04.296172+0000","last_undegraded":"2024-04-26T22:28:04.296172+0000","last_fullsized":"2024-04-26T22:28:04.296172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:07.557474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141923+0000","last_change":"2024-04-26T22:28:02.562956+0000","last_active":"2024-04-26T22:28:04.141923+0000","last_peered":"2024-04-26T22:28:04.141923+0000","last_clean":"2024-04-26T22:28:04.141923+0000","last_became_active":"2024-04-26T22:28:02.562678+0000","last_became_peered":"2024-04-26T22:28:02.562678+0000","last_unstale":"2024-04-26T22:28:04.141923+0000","last_undegraded":"2024-04-26T22:28:04.141923+0000","last_fullsized":"2024-04-26T22:28:04.141923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093122+0000","last_change":"2024-04-26T22:28:01.554918+0000","last_active":"2024-04-26T22:28:04.093122+0000","last_peered":"2024-04-26T22:28:04.093122+0000","last_clean":"2024-04-26T22:28:04.093122+0000","last_became_active":"2024-04-26T22:28:01.554637+0000","last_became_peered":"2024-04-26T22:28:01.554637+0000","last_unstale":"2024-04-26T22:28:04.093122+0000","last_undegraded":"2024-04-26T22:28:04.093122+0000","last_fullsized":"2024-04-26T22:28:04.093122+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142222+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:04.142222+0000","last_peered":"2024-04-26T22:28:04.142222+0000","last_clean":"2024-04-26T22:28:04.142222+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:04.142222+0000","last_undegraded":"2024-04-26T22:28:04.142222+0000","last_fullsized":"2024-04-26T22:28:04.142222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":725149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":134795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":134761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":725059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:11.058 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:11.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.172+0000 7f2d49d78640 1 Processor -- start 2024-04-26T22:28:11.172 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.172+0000 7f2d49d78640 1 -- start start 2024-04-26T22:28:11.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2d44159a90 0x7f2d4406fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.173 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d44070130 0x7f2d4415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44160150 0x7f2d4415c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2d441605c0 con 0x7f2d44070130 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2d44072e30 con 0x7f2d44159a90 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d49d78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d44073110 con 0x7f2d44160150 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d44070130 0x7f2d4415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2d44070130 0x7f2d4415fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43406/0 (socket says 172.21.15.62:43406) 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 -- 172.21.15.62:0/3810307992 learned_addr learned my addr 172.21.15.62:0/3810307992 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d43fff640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44160150 0x7f2d4415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d437fe640 1 --2- 172.21.15.62:0/3810307992 >> v2:172.21.15.174:3300/0 conn(0x7f2d44159a90 0x7f2d4406fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 -- 172.21.15.62:0/3810307992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44160150 msgr2=0x7f2d4415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44160150 0x7f2d4415c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 -- 172.21.15.62:0/3810307992 >> v2:172.21.15.174:3300/0 conn(0x7f2d44159a90 msgr2=0x7f2d4406fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:11.174 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 --2- 172.21.15.62:0/3810307992 >> v2:172.21.15.174:3300/0 conn(0x7f2d44159a90 0x7f2d4406fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.175 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.173+0000 7f2d42ffd640 1 -- 172.21.15.62:0/3810307992 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440733f0 con 0x7f2d44070130 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.174+0000 7f2d42ffd640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44070130 0x7f2d4415fc10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d44074a10 tx=0x7f2d3802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab47eee75241cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.174+0000 7f2d427fc640 1 -- 172.21.15.62:0/3810307992 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38037070 con 0x7f2d44070130 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.174+0000 7f2d427fc640 1 -- 172.21.15.62:0/3810307992 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3802ae00 con 0x7f2d44070130 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.174+0000 7f2d427fc640 1 -- 172.21.15.62:0/3810307992 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38032660 con 0x7f2d44070130 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 -- 172.21.15.62:0/3810307992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44070130 msgr2=0x7f2d4415fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44070130 0x7f2d4415fc10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d44074a10 tx=0x7f2d3802a0b0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.176 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 -- 172.21.15.62:0/3810307992 shutdown_connections 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44160150 0x7f2d4415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 --2- 172.21.15.62:0/3810307992 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44070130 0x7f2d4415fc10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 --2- 172.21.15.62:0/3810307992 >> v2:172.21.15.174:3300/0 conn(0x7f2d44159a90 0x7f2d4406fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 -- 172.21.15.62:0/3810307992 >> 172.21.15.62:0/3810307992 conn(0x7f2d440d0d50 msgr2=0x7f2d440d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 -- 172.21.15.62:0/3810307992 shutdown_connections 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.175+0000 7f2d49d78640 1 -- 172.21.15.62:0/3810307992 wait complete. 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.176+0000 7f2d49d78640 1 Processor -- start 2024-04-26T22:28:11.177 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.176+0000 7f2d49d78640 1 -- start start 2024-04-26T22:28:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d44070130 0x7f2d44143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44159a90 0x7f2d441531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.179 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44159e70 con 0x7f2d44160150 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441609d0 con 0x7f2d44070130 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d49d78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d44159ff0 con 0x7f2d44159a90 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d43fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43410/0 (socket says 172.21.15.62:43410) 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d43fff640 1 -- 172.21.15.62:0/1338720384 learned_addr learned my addr 172.21.15.62:0/1338720384 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d437fe640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d44070130 0x7f2d44143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.177+0000 7f2d42ffd640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44159a90 0x7f2d441531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 -- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44159a90 msgr2=0x7f2d441531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44159a90 0x7f2d441531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 -- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d44070130 msgr2=0x7f2d44143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.180 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d44070130 0x7f2d44143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 -- 172.21.15.62:0/1338720384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d441e5b40 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d43fff640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007e40 tx=0x7f2d3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c014620 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3c019070 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c014a20 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d441e5e20 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.178+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d44143c70 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.179+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4406a360 con 0x7f2d44160150 2024-04-26T22:28:11.181 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.179+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d3c014bc0 con 0x7f2d44160150 2024-04-26T22:28:11.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.180+0000 7f2d2bfff640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 0x7f2d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.180+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f2d3c08c2c0 con 0x7f2d44160150 2024-04-26T22:28:11.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.180+0000 7f2d437fe640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 0x7f2d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.181+0000 7f2d437fe640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 0x7f2d3006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006fd0 tx=0x7f2d34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.184 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.184+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d3c05a0a0 con 0x7f2d44160150 2024-04-26T22:28:11.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.365+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d440d0730 con 0x7f2d3006c8f0 2024-04-26T22:28:11.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.371+0000 7f2d2bfff640 1 -- 172.21.15.62:0/1338720384 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f2d440d0730 con 0x7f2d3006c8f0 2024-04-26T22:28:11.374 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:11.375 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.374+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 msgr2=0x7f2d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 0x7f2d3006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006fd0 tx=0x7f2d34008040 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 msgr2=0x7f2d44153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c007e40 tx=0x7f2d3c00e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 shutdown_connections 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2d3006c8f0 0x7f2d3006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d44160150 0x7f2d44153730 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d44159a90 0x7f2d441531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 --2- 172.21.15.62:0/1338720384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d44070130 0x7f2d44143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 >> 172.21.15.62:0/1338720384 conn(0x7f2d440d0d50 msgr2=0x7f2d440cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 shutdown_connections 2024-04-26T22:28:11.378 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.375+0000 7f2d49d78640 1 -- 172.21.15.62:0/1338720384 wait complete. 2024-04-26T22:28:11.386 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2024-04-26T22:28:09.652473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220136,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754584,"statfs":{"total":773094113280,"available":772868694016,"internally_reserved":0,"allocated":4038656,"data_stored":2263084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001875"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093321+0000","last_change":"2024-04-26T22:28:02.557853+0000","last_active":"2024-04-26T22:28:04.093321+0000","last_peered":"2024-04-26T22:28:04.093321+0000","last_clean":"2024-04-26T22:28:04.093321+0000","last_became_active":"2024-04-26T22:28:02.557547+0000","last_became_peered":"2024-04-26T22:28:02.557547+0000","last_unstale":"2024-04-26T22:28:04.093321+0000","last_undegraded":"2024-04-26T22:28:04.093321+0000","last_fullsized":"2024-04-26T22:28:04.093321+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296667+0000","last_change":"2024-04-26T22:28:02.565136+0000","last_active":"2024-04-26T22:28:04.296667+0000","last_peered":"2024-04-26T22:28:04.296667+0000","last_clean":"2024-04-26T22:28:04.296667+0000","last_became_active":"2024-04-26T22:28:02.564862+0000","last_became_peered":"2024-04-26T22:28:02.564862+0000","last_unstale":"2024-04-26T22:28:04.296667+0000","last_undegraded":"2024-04-26T22:28:04.296667+0000","last_fullsized":"2024-04-26T22:28:04.296667+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:40.654516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561035+0000","last_change":"2024-04-26T22:28:02.568096+0000","last_active":"2024-04-26T22:28:03.561035+0000","last_peered":"2024-04-26T22:28:03.561035+0000","last_clean":"2024-04-26T22:28:03.561035+0000","last_became_active":"2024-04-26T22:28:02.567788+0000","last_became_peered":"2024-04-26T22:28:02.567788+0000","last_unstale":"2024-04-26T22:28:03.561035+0000","last_undegraded":"2024-04-26T22:28:03.561035+0000","last_fullsized":"2024-04-26T22:28:03.561035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:07.397727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559548+0000","last_change":"2024-04-26T22:28:02.563172+0000","last_active":"2024-04-26T22:28:03.559548+0000","last_peered":"2024-04-26T22:28:03.559548+0000","last_clean":"2024-04-26T22:28:03.559548+0000","last_became_active":"2024-04-26T22:28:02.562479+0000","last_became_peered":"2024-04-26T22:28:02.562479+0000","last_unstale":"2024-04-26T22:28:03.559548+0000","last_undegraded":"2024-04-26T22:28:03.559548+0000","last_fullsized":"2024-04-26T22:28:03.559548+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914867+0000","last_change":"2024-04-26T22:28:02.565256+0000","last_active":"2024-04-26T22:28:03.914867+0000","last_peered":"2024-04-26T22:28:03.914867+0000","last_clean":"2024-04-26T22:28:03.914867+0000","last_became_active":"2024-04-26T22:28:02.564971+0000","last_became_peered":"2024-04-26T22:28:02.564971+0000","last_unstale":"2024-04-26T22:28:03.914867+0000","last_undegraded":"2024-04-26T22:28:03.914867+0000","last_fullsized":"2024-04-26T22:28:03.914867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:10.979401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296114+0000","last_change":"2024-04-26T22:28:02.562239+0000","last_active":"2024-04-26T22:28:04.296114+0000","last_peered":"2024-04-26T22:28:04.296114+0000","last_clean":"2024-04-26T22:28:04.296114+0000","last_became_active":"2024-04-26T22:28:02.561843+0000","last_became_peered":"2024-04-26T22:28:02.561843+0000","last_unstale":"2024-04-26T22:28:04.296114+0000","last_undegraded":"2024-04-26T22:28:04.296114+0000","last_fullsized":"2024-04-26T22:28:04.296114+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141972+0000","last_change":"2024-04-26T22:28:02.556548+0000","last_active":"2024-04-26T22:28:04.141972+0000","last_peered":"2024-04-26T22:28:04.141972+0000","last_clean":"2024-04-26T22:28:04.141972+0000","last_became_active":"2024-04-26T22:28:02.556221+0000","last_became_peered":"2024-04-26T22:28:02.556221+0000","last_unstale":"2024-04-26T22:28:04.141972+0000","last_undegraded":"2024-04-26T22:28:04.141972+0000","last_fullsized":"2024-04-26T22:28:04.141972+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:07.397727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141697+0000","last_change":"2024-04-26T22:28:02.564065+0000","last_active":"2024-04-26T22:28:04.141697+0000","last_peered":"2024-04-26T22:28:04.141697+0000","last_clean":"2024-04-26T22:28:04.141697+0000","last_became_active":"2024-04-26T22:28:02.563390+0000","last_became_peered":"2024-04-26T22:28:02.563390+0000","last_unstale":"2024-04-26T22:28:04.141697+0000","last_undegraded":"2024-04-26T22:28:04.141697+0000","last_fullsized":"2024-04-26T22:28:04.141697+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560040+0000","last_change":"2024-04-26T22:28:02.569886+0000","last_active":"2024-04-26T22:28:03.560040+0000","last_peered":"2024-04-26T22:28:03.560040+0000","last_clean":"2024-04-26T22:28:03.560040+0000","last_became_active":"2024-04-26T22:28:02.569468+0000","last_became_peered":"2024-04-26T22:28:02.569468+0000","last_unstale":"2024-04-26T22:28:03.560040+0000","last_undegraded":"2024-04-26T22:28:03.560040+0000","last_fullsized":"2024-04-26T22:28:03.560040+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559451+0000","last_change":"2024-04-26T22:28:02.574061+0000","last_active":"2024-04-26T22:28:03.559451+0000","last_peered":"2024-04-26T22:28:03.559451+0000","last_clean":"2024-04-26T22:28:03.559451+0000","last_became_active":"2024-04-26T22:28:02.573739+0000","last_became_peered":"2024-04-26T22:28:02.573739+0000","last_unstale":"2024-04-26T22:28:03.559451+0000","last_undegraded":"2024-04-26T22:28:03.559451+0000","last_fullsized":"2024-04-26T22:28:03.559451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141961+0000","last_change":"2024-04-26T22:28:02.570383+0000","last_active":"2024-04-26T22:28:04.141961+0000","last_peered":"2024-04-26T22:28:04.141961+0000","last_clean":"2024-04-26T22:28:04.141961+0000","last_became_active":"2024-04-26T22:28:02.570047+0000","last_became_peered":"2024-04-26T22:28:02.570047+0000","last_unstale":"2024-04-26T22:28:04.141961+0000","last_undegraded":"2024-04-26T22:28:04.141961+0000","last_fullsized":"2024-04-26T22:28:04.141961+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560841+0000","last_change":"2024-04-26T22:28:02.572697+0000","last_active":"2024-04-26T22:28:03.560841+0000","last_peered":"2024-04-26T22:28:03.560841+0000","last_clean":"2024-04-26T22:28:03.560841+0000","last_became_active":"2024-04-26T22:28:02.572395+0000","last_became_peered":"2024-04-26T22:28:02.572395+0000","last_unstale":"2024-04-26T22:28:03.560841+0000","last_undegraded":"2024-04-26T22:28:03.560841+0000","last_fullsized":"2024-04-26T22:28:03.560841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:13.832773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914855+0000","last_change":"2024-04-26T22:28:02.559102+0000","last_active":"2024-04-26T22:28:03.914855+0000","last_peered":"2024-04-26T22:28:03.914855+0000","last_clean":"2024-04-26T22:28:03.914855+0000","last_became_active":"2024-04-26T22:28:02.558834+0000","last_became_peered":"2024-04-26T22:28:02.558834+0000","last_unstale":"2024-04-26T22:28:03.914855+0000","last_undegraded":"2024-04-26T22:28:03.914855+0000","last_fullsized":"2024-04-26T22:28:03.914855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:44.309166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296202+0000","last_change":"2024-04-26T22:28:02.563547+0000","last_active":"2024-04-26T22:28:04.296202+0000","last_peered":"2024-04-26T22:28:04.296202+0000","last_clean":"2024-04-26T22:28:04.296202+0000","last_became_active":"2024-04-26T22:28:02.563212+0000","last_became_peered":"2024-04-26T22:28:02.563212+0000","last_unstale":"2024-04-26T22:28:04.296202+0000","last_undegraded":"2024-04-26T22:28:04.296202+0000","last_fullsized":"2024-04-26T22:28:04.296202+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.992166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560537+0000","last_change":"2024-04-26T22:28:02.554993+0000","last_active":"2024-04-26T22:28:03.560537+0000","last_peered":"2024-04-26T22:28:03.560537+0000","last_clean":"2024-04-26T22:28:03.560537+0000","last_became_active":"2024-04-26T22:28:02.554174+0000","last_became_peered":"2024-04-26T22:28:02.554174+0000","last_unstale":"2024-04-26T22:28:03.560537+0000","last_undegraded":"2024-04-26T22:28:03.560537+0000","last_fullsized":"2024-04-26T22:28:03.560537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142255+0000","last_change":"2024-04-26T22:28:02.565230+0000","last_active":"2024-04-26T22:28:04.142255+0000","last_peered":"2024-04-26T22:28:04.142255+0000","last_clean":"2024-04-26T22:28:04.142255+0000","last_became_active":"2024-04-26T22:28:02.564938+0000","last_became_peered":"2024-04-26T22:28:02.564938+0000","last_unstale":"2024-04-26T22:28:04.142255+0000","last_undegraded":"2024-04-26T22:28:04.142255+0000","last_fullsized":"2024-04-26T22:28:04.142255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142319+0000","last_change":"2024-04-26T22:28:02.557007+0000","last_active":"2024-04-26T22:28:04.142319+0000","last_peered":"2024-04-26T22:28:04.142319+0000","last_clean":"2024-04-26T22:28:04.142319+0000","last_became_active":"2024-04-26T22:28:02.556160+0000","last_became_peered":"2024-04-26T22:28:02.556160+0000","last_unstale":"2024-04-26T22:28:04.142319+0000","last_undegraded":"2024-04-26T22:28:04.142319+0000","last_fullsized":"2024-04-26T22:28:04.142319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.992166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296662+0000","last_change":"2024-04-26T22:28:02.571421+0000","last_active":"2024-04-26T22:28:04.296662+0000","last_peered":"2024-04-26T22:28:04.296662+0000","last_clean":"2024-04-26T22:28:04.296662+0000","last_became_active":"2024-04-26T22:28:02.571057+0000","last_became_peered":"2024-04-26T22:28:02.571057+0000","last_unstale":"2024-04-26T22:28:04.296662+0000","last_undegraded":"2024-04-26T22:28:04.296662+0000","last_fullsized":"2024-04-26T22:28:04.296662+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:11.270062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296381+0000","last_change":"2024-04-26T22:28:02.572507+0000","last_active":"2024-04-26T22:28:04.296381+0000","last_peered":"2024-04-26T22:28:04.296381+0000","last_clean":"2024-04-26T22:28:04.296381+0000","last_became_active":"2024-04-26T22:28:02.572194+0000","last_became_peered":"2024-04-26T22:28:02.572194+0000","last_unstale":"2024-04-26T22:28:04.296381+0000","last_undegraded":"2024-04-26T22:28:04.296381+0000","last_fullsized":"2024-04-26T22:28:04.296381+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:05.320316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915010+0000","last_change":"2024-04-26T22:28:02.556476+0000","last_active":"2024-04-26T22:28:03.915010+0000","last_peered":"2024-04-26T22:28:03.915010+0000","last_clean":"2024-04-26T22:28:03.915010+0000","last_became_active":"2024-04-26T22:28:02.556191+0000","last_became_peered":"2024-04-26T22:28:02.556191+0000","last_unstale":"2024-04-26T22:28:03.915010+0000","last_undegraded":"2024-04-26T22:28:03.915010+0000","last_fullsized":"2024-04-26T22:28:03.915010+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:08.189558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561210+0000","last_change":"2024-04-26T22:28:02.552028+0000","last_active":"2024-04-26T22:28:03.561210+0000","last_peered":"2024-04-26T22:28:03.561210+0000","last_clean":"2024-04-26T22:28:03.561210+0000","last_became_active":"2024-04-26T22:28:02.551742+0000","last_became_peered":"2024-04-26T22:28:02.551742+0000","last_unstale":"2024-04-26T22:28:03.561210+0000","last_undegraded":"2024-04-26T22:28:03.561210+0000","last_fullsized":"2024-04-26T22:28:03.561210+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560233+0000","last_change":"2024-04-26T22:28:02.562010+0000","last_active":"2024-04-26T22:28:03.560233+0000","last_peered":"2024-04-26T22:28:03.560233+0000","last_clean":"2024-04-26T22:28:03.560233+0000","last_became_active":"2024-04-26T22:28:02.561719+0000","last_became_peered":"2024-04-26T22:28:02.561719+0000","last_unstale":"2024-04-26T22:28:03.560233+0000","last_undegraded":"2024-04-26T22:28:03.560233+0000","last_fullsized":"2024-04-26T22:28:03.560233+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554272+0000","last_change":"2024-04-26T22:28:02.554165+0000","last_active":"2024-04-26T22:28:03.554272+0000","last_peered":"2024-04-26T22:28:03.554272+0000","last_clean":"2024-04-26T22:28:03.554272+0000","last_became_active":"2024-04-26T22:28:02.553872+0000","last_became_peered":"2024-04-26T22:28:02.553872+0000","last_unstale":"2024-04-26T22:28:03.554272+0000","last_undegraded":"2024-04-26T22:28:03.554272+0000","last_fullsized":"2024-04-26T22:28:03.554272+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559588+0000","last_change":"2024-04-26T22:28:02.558912+0000","last_active":"2024-04-26T22:28:03.559588+0000","last_peered":"2024-04-26T22:28:03.559588+0000","last_clean":"2024-04-26T22:28:03.559588+0000","last_became_active":"2024-04-26T22:28:02.558633+0000","last_became_peered":"2024-04-26T22:28:02.558633+0000","last_unstale":"2024-04-26T22:28:03.559588+0000","last_undegraded":"2024-04-26T22:28:03.559588+0000","last_fullsized":"2024-04-26T22:28:03.559588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559564+0000","last_change":"2024-04-26T22:28:02.573937+0000","last_active":"2024-04-26T22:28:03.559564+0000","last_peered":"2024-04-26T22:28:03.559564+0000","last_clean":"2024-04-26T22:28:03.559564+0000","last_became_active":"2024-04-26T22:28:02.573553+0000","last_became_peered":"2024-04-26T22:28:02.573553+0000","last_unstale":"2024-04-26T22:28:03.559564+0000","last_undegraded":"2024-04-26T22:28:03.559564+0000","last_fullsized":"2024-04-26T22:28:03.559564+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296785+0000","last_change":"2024-04-26T22:28:02.565075+0000","last_active":"2024-04-26T22:28:04.296785+0000","last_peered":"2024-04-26T22:28:04.296785+0000","last_clean":"2024-04-26T22:28:04.296785+0000","last_became_active":"2024-04-26T22:28:02.564553+0000","last_became_peered":"2024-04-26T22:28:02.564553+0000","last_unstale":"2024-04-26T22:28:04.296785+0000","last_undegraded":"2024-04-26T22:28:04.296785+0000","last_fullsized":"2024-04-26T22:28:04.296785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:11.425121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559403+0000","last_change":"2024-04-26T22:28:02.568815+0000","last_active":"2024-04-26T22:28:03.559403+0000","last_peered":"2024-04-26T22:28:03.559403+0000","last_clean":"2024-04-26T22:28:03.559403+0000","last_became_active":"2024-04-26T22:28:02.568165+0000","last_became_peered":"2024-04-26T22:28:02.568165+0000","last_unstale":"2024-04-26T22:28:03.559403+0000","last_undegraded":"2024-04-26T22:28:03.559403+0000","last_fullsized":"2024-04-26T22:28:03.559403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561227+0000","last_change":"2024-04-26T22:28:02.544685+0000","last_active":"2024-04-26T22:28:03.561227+0000","last_peered":"2024-04-26T22:28:03.561227+0000","last_clean":"2024-04-26T22:28:03.561227+0000","last_became_active":"2024-04-26T22:28:02.544398+0000","last_became_peered":"2024-04-26T22:28:02.544398+0000","last_unstale":"2024-04-26T22:28:03.561227+0000","last_undegraded":"2024-04-26T22:28:03.561227+0000","last_fullsized":"2024-04-26T22:28:03.561227+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:55:36.351126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561298+0000","last_change":"2024-04-26T22:28:02.565191+0000","last_active":"2024-04-26T22:28:03.561298+0000","last_peered":"2024-04-26T22:28:03.561298+0000","last_clean":"2024-04-26T22:28:03.561298+0000","last_became_active":"2024-04-26T22:28:02.564949+0000","last_became_peered":"2024-04-26T22:28:02.564949+0000","last_unstale":"2024-04-26T22:28:03.561298+0000","last_undegraded":"2024-04-26T22:28:03.561298+0000","last_fullsized":"2024-04-26T22:28:03.561298+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141913+0000","last_change":"2024-04-26T22:28:02.555628+0000","last_active":"2024-04-26T22:28:04.141913+0000","last_peered":"2024-04-26T22:28:04.141913+0000","last_clean":"2024-04-26T22:28:04.141913+0000","last_became_active":"2024-04-26T22:28:02.555353+0000","last_became_peered":"2024-04-26T22:28:02.555353+0000","last_unstale":"2024-04-26T22:28:04.141913+0000","last_undegraded":"2024-04-26T22:28:04.141913+0000","last_fullsized":"2024-04-26T22:28:04.141913+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:14.642390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093106+0000","last_change":"2024-04-26T22:28:02.564462+0000","last_active":"2024-04-26T22:28:04.093106+0000","last_peered":"2024-04-26T22:28:04.093106+0000","last_clean":"2024-04-26T22:28:04.093106+0000","last_became_active":"2024-04-26T22:28:02.563780+0000","last_became_peered":"2024-04-26T22:28:02.563780+0000","last_unstale":"2024-04-26T22:28:04.093106+0000","last_undegraded":"2024-04-26T22:28:04.093106+0000","last_fullsized":"2024-04-26T22:28:04.093106+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.595312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141684+0000","last_change":"2024-04-26T22:28:02.560240+0000","last_active":"2024-04-26T22:28:04.141684+0000","last_peered":"2024-04-26T22:28:04.141684+0000","last_clean":"2024-04-26T22:28:04.141684+0000","last_became_active":"2024-04-26T22:28:02.559907+0000","last_became_peered":"2024-04-26T22:28:02.559907+0000","last_unstale":"2024-04-26T22:28:04.141684+0000","last_undegraded":"2024-04-26T22:28:04.141684+0000","last_fullsized":"2024-04-26T22:28:04.141684+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554421+0000","last_change":"2024-04-26T22:28:02.554463+0000","last_active":"2024-04-26T22:28:03.554421+0000","last_peered":"2024-04-26T22:28:03.554421+0000","last_clean":"2024-04-26T22:28:03.554421+0000","last_became_active":"2024-04-26T22:28:02.553837+0000","last_became_peered":"2024-04-26T22:28:02.553837+0000","last_unstale":"2024-04-26T22:28:03.554421+0000","last_undegraded":"2024-04-26T22:28:03.554421+0000","last_fullsized":"2024-04-26T22:28:03.554421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:41.593505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093407+0000","last_change":"2024-04-26T22:28:01.551148+0000","last_active":"2024-04-26T22:28:04.093407+0000","last_peered":"2024-04-26T22:28:04.093407+0000","last_clean":"2024-04-26T22:28:04.093407+0000","last_became_active":"2024-04-26T22:28:01.550854+0000","last_became_peered":"2024-04-26T22:28:01.550854+0000","last_unstale":"2024-04-26T22:28:04.093407+0000","last_undegraded":"2024-04-26T22:28:04.093407+0000","last_fullsized":"2024-04-26T22:28:04.093407+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093370+0000","last_change":"2024-04-26T22:28:02.553397+0000","last_active":"2024-04-26T22:28:04.093370+0000","last_peered":"2024-04-26T22:28:04.093370+0000","last_clean":"2024-04-26T22:28:04.093370+0000","last_became_active":"2024-04-26T22:28:02.553087+0000","last_became_peered":"2024-04-26T22:28:02.553087+0000","last_unstale":"2024-04-26T22:28:04.093370+0000","last_undegraded":"2024-04-26T22:28:04.093370+0000","last_fullsized":"2024-04-26T22:28:04.093370+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554403+0000","last_change":"2024-04-26T22:28:01.553286+0000","last_active":"2024-04-26T22:28:03.554403+0000","last_peered":"2024-04-26T22:28:03.554403+0000","last_clean":"2024-04-26T22:28:03.554403+0000","last_became_active":"2024-04-26T22:28:01.552926+0000","last_became_peered":"2024-04-26T22:28:01.552926+0000","last_unstale":"2024-04-26T22:28:03.554403+0000","last_undegraded":"2024-04-26T22:28:03.554403+0000","last_fullsized":"2024-04-26T22:28:03.554403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560171+0000","last_change":"2024-04-26T22:28:02.569421+0000","last_active":"2024-04-26T22:28:03.560171+0000","last_peered":"2024-04-26T22:28:03.560171+0000","last_clean":"2024-04-26T22:28:03.560171+0000","last_became_active":"2024-04-26T22:28:02.569122+0000","last_became_peered":"2024-04-26T22:28:02.569122+0000","last_unstale":"2024-04-26T22:28:03.560171+0000","last_undegraded":"2024-04-26T22:28:03.560171+0000","last_fullsized":"2024-04-26T22:28:03.560171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:55.085718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296345+0000","last_change":"2024-04-26T22:28:01.559844+0000","last_active":"2024-04-26T22:28:04.296345+0000","last_peered":"2024-04-26T22:28:04.296345+0000","last_clean":"2024-04-26T22:28:04.296345+0000","last_became_active":"2024-04-26T22:28:01.559546+0000","last_became_peered":"2024-04-26T22:28:01.559546+0000","last_unstale":"2024-04-26T22:28:04.296345+0000","last_undegraded":"2024-04-26T22:28:04.296345+0000","last_fullsized":"2024-04-26T22:28:04.296345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:20.657589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559452+0000","last_change":"2024-04-26T22:28:02.568495+0000","last_active":"2024-04-26T22:28:03.559452+0000","last_peered":"2024-04-26T22:28:03.559452+0000","last_clean":"2024-04-26T22:28:03.559452+0000","last_became_active":"2024-04-26T22:28:02.567595+0000","last_became_peered":"2024-04-26T22:28:02.567595+0000","last_unstale":"2024-04-26T22:28:03.559452+0000","last_undegraded":"2024-04-26T22:28:03.559452+0000","last_fullsized":"2024-04-26T22:28:03.559452+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915031+0000","last_change":"2024-04-26T22:28:01.552326+0000","last_active":"2024-04-26T22:28:03.915031+0000","last_peered":"2024-04-26T22:28:03.915031+0000","last_clean":"2024-04-26T22:28:03.915031+0000","last_became_active":"2024-04-26T22:28:01.551423+0000","last_became_peered":"2024-04-26T22:28:01.551423+0000","last_unstale":"2024-04-26T22:28:03.915031+0000","last_undegraded":"2024-04-26T22:28:03.915031+0000","last_fullsized":"2024-04-26T22:28:03.915031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:48:41.694091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142012+0000","last_change":"2024-04-26T22:28:02.560673+0000","last_active":"2024-04-26T22:28:04.142012+0000","last_peered":"2024-04-26T22:28:04.142012+0000","last_clean":"2024-04-26T22:28:04.142012+0000","last_became_active":"2024-04-26T22:28:02.560395+0000","last_became_peered":"2024-04-26T22:28:02.560395+0000","last_unstale":"2024-04-26T22:28:04.142012+0000","last_undegraded":"2024-04-26T22:28:04.142012+0000","last_fullsized":"2024-04-26T22:28:04.142012+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561334+0000","last_change":"2024-04-26T22:28:01.549390+0000","last_active":"2024-04-26T22:28:03.561334+0000","last_peered":"2024-04-26T22:28:03.561334+0000","last_clean":"2024-04-26T22:28:03.561334+0000","last_became_active":"2024-04-26T22:28:01.548063+0000","last_became_peered":"2024-04-26T22:28:01.548063+0000","last_unstale":"2024-04-26T22:28:03.561334+0000","last_undegraded":"2024-04-26T22:28:03.561334+0000","last_fullsized":"2024-04-26T22:28:03.561334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:35:07.182821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560519+0000","last_change":"2024-04-26T22:28:02.544922+0000","last_active":"2024-04-26T22:28:03.560519+0000","last_peered":"2024-04-26T22:28:03.560519+0000","last_clean":"2024-04-26T22:28:03.560519+0000","last_became_active":"2024-04-26T22:28:02.544326+0000","last_became_peered":"2024-04-26T22:28:02.544326+0000","last_unstale":"2024-04-26T22:28:03.560519+0000","last_undegraded":"2024-04-26T22:28:03.560519+0000","last_fullsized":"2024-04-26T22:28:03.560519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560139+0000","last_change":"2024-04-26T22:28:01.548404+0000","last_active":"2024-04-26T22:28:03.560139+0000","last_peered":"2024-04-26T22:28:03.560139+0000","last_clean":"2024-04-26T22:28:03.560139+0000","last_became_active":"2024-04-26T22:28:01.548088+0000","last_became_peered":"2024-04-26T22:28:01.548088+0000","last_unstale":"2024-04-26T22:28:03.560139+0000","last_undegraded":"2024-04-26T22:28:03.560139+0000","last_fullsized":"2024-04-26T22:28:03.560139+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:28.781594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559971+0000","last_change":"2024-04-26T22:28:02.554967+0000","last_active":"2024-04-26T22:28:03.559971+0000","last_peered":"2024-04-26T22:28:03.559971+0000","last_clean":"2024-04-26T22:28:03.559971+0000","last_became_active":"2024-04-26T22:28:02.554198+0000","last_became_peered":"2024-04-26T22:28:02.554198+0000","last_unstale":"2024-04-26T22:28:03.559971+0000","last_undegraded":"2024-04-26T22:28:03.559971+0000","last_fullsized":"2024-04-26T22:28:03.559971+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296024+0000","last_change":"2024-04-26T22:28:01.556939+0000","last_active":"2024-04-26T22:28:04.296024+0000","last_peered":"2024-04-26T22:28:04.296024+0000","last_clean":"2024-04-26T22:28:04.296024+0000","last_became_active":"2024-04-26T22:28:01.556663+0000","last_became_peered":"2024-04-26T22:28:01.556663+0000","last_unstale":"2024-04-26T22:28:04.296024+0000","last_undegraded":"2024-04-26T22:28:04.296024+0000","last_fullsized":"2024-04-26T22:28:04.296024+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:23.588575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559050+0000","last_change":"2024-04-26T22:28:02.561770+0000","last_active":"2024-04-26T22:28:03.559050+0000","last_peered":"2024-04-26T22:28:03.559050+0000","last_clean":"2024-04-26T22:28:03.559050+0000","last_became_active":"2024-04-26T22:28:02.561463+0000","last_became_peered":"2024-04-26T22:28:02.561463+0000","last_unstale":"2024-04-26T22:28:03.559050+0000","last_undegraded":"2024-04-26T22:28:03.559050+0000","last_fullsized":"2024-04-26T22:28:03.559050+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:09:55.085718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559920+0000","last_change":"2024-04-26T22:28:01.552353+0000","last_active":"2024-04-26T22:28:03.559920+0000","last_peered":"2024-04-26T22:28:03.559920+0000","last_clean":"2024-04-26T22:28:03.559920+0000","last_became_active":"2024-04-26T22:28:01.549925+0000","last_became_peered":"2024-04-26T22:28:01.549925+0000","last_unstale":"2024-04-26T22:28:03.559920+0000","last_undegraded":"2024-04-26T22:28:03.559920+0000","last_fullsized":"2024-04-26T22:28:03.559920+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:47.117169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560778+0000","last_change":"2024-04-26T22:28:02.566415+0000","last_active":"2024-04-26T22:28:03.560778+0000","last_peered":"2024-04-26T22:28:03.560778+0000","last_clean":"2024-04-26T22:28:03.560778+0000","last_became_active":"2024-04-26T22:28:02.566124+0000","last_became_peered":"2024-04-26T22:28:02.566124+0000","last_unstale":"2024-04-26T22:28:03.560778+0000","last_undegraded":"2024-04-26T22:28:03.560778+0000","last_fullsized":"2024-04-26T22:28:03.560778+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296623+0000","last_change":"2024-04-26T22:28:01.555381+0000","last_active":"2024-04-26T22:28:04.296623+0000","last_peered":"2024-04-26T22:28:04.296623+0000","last_clean":"2024-04-26T22:28:04.296623+0000","last_became_active":"2024-04-26T22:28:01.555132+0000","last_became_peered":"2024-04-26T22:28:01.555132+0000","last_unstale":"2024-04-26T22:28:04.296623+0000","last_undegraded":"2024-04-26T22:28:04.296623+0000","last_fullsized":"2024-04-26T22:28:04.296623+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:40.586767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296660+0000","last_change":"2024-04-26T22:28:02.572112+0000","last_active":"2024-04-26T22:28:04.296660+0000","last_peered":"2024-04-26T22:28:04.296660+0000","last_clean":"2024-04-26T22:28:04.296660+0000","last_became_active":"2024-04-26T22:28:02.571821+0000","last_became_peered":"2024-04-26T22:28:02.571821+0000","last_unstale":"2024-04-26T22:28:04.296660+0000","last_undegraded":"2024-04-26T22:28:04.296660+0000","last_fullsized":"2024-04-26T22:28:04.296660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:08.426549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554484+0000","last_change":"2024-04-26T22:28:01.563856+0000","last_active":"2024-04-26T22:28:03.554484+0000","last_peered":"2024-04-26T22:28:03.554484+0000","last_clean":"2024-04-26T22:28:03.554484+0000","last_became_active":"2024-04-26T22:28:01.563514+0000","last_became_peered":"2024-04-26T22:28:01.563514+0000","last_unstale":"2024-04-26T22:28:03.554484+0000","last_undegraded":"2024-04-26T22:28:03.554484+0000","last_fullsized":"2024-04-26T22:28:03.554484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296371+0000","last_change":"2024-04-26T22:28:02.566484+0000","last_active":"2024-04-26T22:28:04.296371+0000","last_peered":"2024-04-26T22:28:04.296371+0000","last_clean":"2024-04-26T22:28:04.296371+0000","last_became_active":"2024-04-26T22:28:02.565731+0000","last_became_peered":"2024-04-26T22:28:02.565731+0000","last_unstale":"2024-04-26T22:28:04.296371+0000","last_undegraded":"2024-04-26T22:28:04.296371+0000","last_fullsized":"2024-04-26T22:28:04.296371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:43.011198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561005+0000","last_change":"2024-04-26T22:28:01.553958+0000","last_active":"2024-04-26T22:28:03.561005+0000","last_peered":"2024-04-26T22:28:03.561005+0000","last_clean":"2024-04-26T22:28:03.561005+0000","last_became_active":"2024-04-26T22:28:01.553552+0000","last_became_peered":"2024-04-26T22:28:01.553552+0000","last_unstale":"2024-04-26T22:28:03.561005+0000","last_undegraded":"2024-04-26T22:28:03.561005+0000","last_fullsized":"2024-04-26T22:28:03.561005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560821+0000","last_change":"2024-04-26T22:28:02.568051+0000","last_active":"2024-04-26T22:28:03.560821+0000","last_peered":"2024-04-26T22:28:03.560821+0000","last_clean":"2024-04-26T22:28:03.560821+0000","last_became_active":"2024-04-26T22:28:02.567797+0000","last_became_peered":"2024-04-26T22:28:02.567797+0000","last_unstale":"2024-04-26T22:28:03.560821+0000","last_undegraded":"2024-04-26T22:28:03.560821+0000","last_fullsized":"2024-04-26T22:28:03.560821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296616+0000","last_change":"2024-04-26T22:28:01.559144+0000","last_active":"2024-04-26T22:28:04.296616+0000","last_peered":"2024-04-26T22:28:04.296616+0000","last_clean":"2024-04-26T22:28:04.296616+0000","last_became_active":"2024-04-26T22:28:01.558496+0000","last_became_peered":"2024-04-26T22:28:01.558496+0000","last_unstale":"2024-04-26T22:28:04.296616+0000","last_undegraded":"2024-04-26T22:28:04.296616+0000","last_fullsized":"2024-04-26T22:28:04.296616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:04.318619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559095+0000","last_change":"2024-04-26T22:28:02.567564+0000","last_active":"2024-04-26T22:28:03.559095+0000","last_peered":"2024-04-26T22:28:03.559095+0000","last_clean":"2024-04-26T22:28:03.559095+0000","last_became_active":"2024-04-26T22:28:02.567232+0000","last_became_peered":"2024-04-26T22:28:02.567232+0000","last_unstale":"2024-04-26T22:28:03.559095+0000","last_undegraded":"2024-04-26T22:28:03.559095+0000","last_fullsized":"2024-04-26T22:28:03.559095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:19.721382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561409+0000","last_change":"2024-04-26T22:28:01.550978+0000","last_active":"2024-04-26T22:28:03.561409+0000","last_peered":"2024-04-26T22:28:03.561409+0000","last_clean":"2024-04-26T22:28:03.561409+0000","last_became_active":"2024-04-26T22:28:01.550154+0000","last_became_peered":"2024-04-26T22:28:01.550154+0000","last_unstale":"2024-04-26T22:28:03.561409+0000","last_undegraded":"2024-04-26T22:28:03.561409+0000","last_fullsized":"2024-04-26T22:28:03.561409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:09.972664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296276+0000","last_change":"2024-04-26T22:28:02.562944+0000","last_active":"2024-04-26T22:28:04.296276+0000","last_peered":"2024-04-26T22:28:04.296276+0000","last_clean":"2024-04-26T22:28:04.296276+0000","last_became_active":"2024-04-26T22:28:02.562580+0000","last_became_peered":"2024-04-26T22:28:02.562580+0000","last_unstale":"2024-04-26T22:28:04.296276+0000","last_undegraded":"2024-04-26T22:28:04.296276+0000","last_fullsized":"2024-04-26T22:28:04.296276+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:32.066791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559152+0000","last_change":"2024-04-26T22:28:01.551899+0000","last_active":"2024-04-26T22:28:03.559152+0000","last_peered":"2024-04-26T22:28:03.559152+0000","last_clean":"2024-04-26T22:28:03.559152+0000","last_became_active":"2024-04-26T22:28:01.551587+0000","last_became_peered":"2024-04-26T22:28:01.551587+0000","last_unstale":"2024-04-26T22:28:03.559152+0000","last_undegraded":"2024-04-26T22:28:03.559152+0000","last_fullsized":"2024-04-26T22:28:03.559152+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142081+0000","last_change":"2024-04-26T22:28:02.560842+0000","last_active":"2024-04-26T22:28:04.142081+0000","last_peered":"2024-04-26T22:28:04.142081+0000","last_clean":"2024-04-26T22:28:04.142081+0000","last_became_active":"2024-04-26T22:28:02.560572+0000","last_became_peered":"2024-04-26T22:28:02.560572+0000","last_unstale":"2024-04-26T22:28:04.142081+0000","last_undegraded":"2024-04-26T22:28:04.142081+0000","last_fullsized":"2024-04-26T22:28:04.142081+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296115+0000","last_change":"2024-04-26T22:28:01.555910+0000","last_active":"2024-04-26T22:28:04.296115+0000","last_peered":"2024-04-26T22:28:04.296115+0000","last_clean":"2024-04-26T22:28:04.296115+0000","last_became_active":"2024-04-26T22:28:01.555596+0000","last_became_peered":"2024-04-26T22:28:01.555596+0000","last_unstale":"2024-04-26T22:28:04.296115+0000","last_undegraded":"2024-04-26T22:28:04.296115+0000","last_fullsized":"2024-04-26T22:28:04.296115+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:35.686888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093195+0000","last_change":"2024-04-26T22:28:02.559888+0000","last_active":"2024-04-26T22:28:04.093195+0000","last_peered":"2024-04-26T22:28:04.093195+0000","last_clean":"2024-04-26T22:28:04.093195+0000","last_became_active":"2024-04-26T22:28:02.559626+0000","last_became_peered":"2024-04-26T22:28:02.559626+0000","last_unstale":"2024-04-26T22:28:04.093195+0000","last_undegraded":"2024-04-26T22:28:04.093195+0000","last_fullsized":"2024-04-26T22:28:04.093195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296087+0000","last_change":"2024-04-26T22:28:01.559695+0000","last_active":"2024-04-26T22:28:04.296087+0000","last_peered":"2024-04-26T22:28:04.296087+0000","last_clean":"2024-04-26T22:28:04.296087+0000","last_became_active":"2024-04-26T22:28:01.559448+0000","last_became_peered":"2024-04-26T22:28:01.559448+0000","last_unstale":"2024-04-26T22:28:04.296087+0000","last_undegraded":"2024-04-26T22:28:04.296087+0000","last_fullsized":"2024-04-26T22:28:04.296087+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:17.137411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559868+0000","last_change":"2024-04-26T22:28:02.553976+0000","last_active":"2024-04-26T22:28:03.559868+0000","last_peered":"2024-04-26T22:28:03.559868+0000","last_clean":"2024-04-26T22:28:03.559868+0000","last_became_active":"2024-04-26T22:28:02.553134+0000","last_became_peered":"2024-04-26T22:28:02.553134+0000","last_unstale":"2024-04-26T22:28:03.559868+0000","last_undegraded":"2024-04-26T22:28:03.559868+0000","last_fullsized":"2024-04-26T22:28:03.559868+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:27:36.693625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559673+0000","last_change":"2024-04-26T22:28:01.560134+0000","last_active":"2024-04-26T22:28:03.559673+0000","last_peered":"2024-04-26T22:28:03.559673+0000","last_clean":"2024-04-26T22:28:03.559673+0000","last_became_active":"2024-04-26T22:28:01.559832+0000","last_became_peered":"2024-04-26T22:28:01.559832+0000","last_unstale":"2024-04-26T22:28:03.559673+0000","last_undegraded":"2024-04-26T22:28:03.559673+0000","last_fullsized":"2024-04-26T22:28:03.559673+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:57.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561020+0000","last_change":"2024-04-26T22:28:02.549060+0000","last_active":"2024-04-26T22:28:03.561020+0000","last_peered":"2024-04-26T22:28:03.561020+0000","last_clean":"2024-04-26T22:28:03.561020+0000","last_became_active":"2024-04-26T22:28:02.548805+0000","last_became_peered":"2024-04-26T22:28:02.548805+0000","last_unstale":"2024-04-26T22:28:03.561020+0000","last_undegraded":"2024-04-26T22:28:03.561020+0000","last_fullsized":"2024-04-26T22:28:03.561020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:53.571447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559988+0000","last_change":"2024-04-26T22:28:01.550770+0000","last_active":"2024-04-26T22:28:03.559988+0000","last_peered":"2024-04-26T22:28:03.559988+0000","last_clean":"2024-04-26T22:28:03.559988+0000","last_became_active":"2024-04-26T22:28:01.550490+0000","last_became_peered":"2024-04-26T22:28:01.550490+0000","last_unstale":"2024-04-26T22:28:03.559988+0000","last_undegraded":"2024-04-26T22:28:03.559988+0000","last_fullsized":"2024-04-26T22:28:03.559988+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:43.302429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559225+0000","last_change":"2024-04-26T22:28:02.562897+0000","last_active":"2024-04-26T22:28:03.559225+0000","last_peered":"2024-04-26T22:28:03.559225+0000","last_clean":"2024-04-26T22:28:03.559225+0000","last_became_active":"2024-04-26T22:28:02.562115+0000","last_became_peered":"2024-04-26T22:28:02.562115+0000","last_unstale":"2024-04-26T22:28:03.559225+0000","last_undegraded":"2024-04-26T22:28:03.559225+0000","last_fullsized":"2024-04-26T22:28:03.559225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:56.550137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560102+0000","last_change":"2024-04-26T22:28:01.548123+0000","last_active":"2024-04-26T22:28:03.560102+0000","last_peered":"2024-04-26T22:28:03.560102+0000","last_clean":"2024-04-26T22:28:03.560102+0000","last_became_active":"2024-04-26T22:28:01.547847+0000","last_became_peered":"2024-04-26T22:28:01.547847+0000","last_unstale":"2024-04-26T22:28:03.560102+0000","last_undegraded":"2024-04-26T22:28:03.560102+0000","last_fullsized":"2024-04-26T22:28:03.560102+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915066+0000","last_change":"2024-04-26T22:28:02.556960+0000","last_active":"2024-04-26T22:28:03.915066+0000","last_peered":"2024-04-26T22:28:03.915066+0000","last_clean":"2024-04-26T22:28:03.915066+0000","last_became_active":"2024-04-26T22:28:02.556640+0000","last_became_peered":"2024-04-26T22:28:02.556640+0000","last_unstale":"2024-04-26T22:28:03.915066+0000","last_undegraded":"2024-04-26T22:28:03.915066+0000","last_fullsized":"2024-04-26T22:28:03.915066+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:37.889245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560209+0000","last_change":"2024-04-26T22:28:01.548757+0000","last_active":"2024-04-26T22:28:03.560209+0000","last_peered":"2024-04-26T22:28:03.560209+0000","last_clean":"2024-04-26T22:28:03.560209+0000","last_became_active":"2024-04-26T22:28:01.548498+0000","last_became_peered":"2024-04-26T22:28:01.548498+0000","last_unstale":"2024-04-26T22:28:03.560209+0000","last_undegraded":"2024-04-26T22:28:03.560209+0000","last_fullsized":"2024-04-26T22:28:03.560209+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:09.306834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914858+0000","last_change":"2024-04-26T22:28:02.560084+0000","last_active":"2024-04-26T22:28:03.914858+0000","last_peered":"2024-04-26T22:28:03.914858+0000","last_clean":"2024-04-26T22:28:03.914858+0000","last_became_active":"2024-04-26T22:28:02.559791+0000","last_became_peered":"2024-04-26T22:28:02.559791+0000","last_unstale":"2024-04-26T22:28:03.914858+0000","last_undegraded":"2024-04-26T22:28:03.914858+0000","last_fullsized":"2024-04-26T22:28:03.914858+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:48.123906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554539+0000","last_change":"2024-04-26T22:28:01.559890+0000","last_active":"2024-04-26T22:28:03.554539+0000","last_peered":"2024-04-26T22:28:03.554539+0000","last_clean":"2024-04-26T22:28:03.554539+0000","last_became_active":"2024-04-26T22:28:01.559564+0000","last_became_peered":"2024-04-26T22:28:01.559564+0000","last_unstale":"2024-04-26T22:28:03.554539+0000","last_undegraded":"2024-04-26T22:28:03.554539+0000","last_fullsized":"2024-04-26T22:28:03.554539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915035+0000","last_change":"2024-04-26T22:28:02.563648+0000","last_active":"2024-04-26T22:28:03.915035+0000","last_peered":"2024-04-26T22:28:03.915035+0000","last_clean":"2024-04-26T22:28:03.915035+0000","last_became_active":"2024-04-26T22:28:02.562244+0000","last_became_peered":"2024-04-26T22:28:02.562244+0000","last_unstale":"2024-04-26T22:28:03.915035+0000","last_undegraded":"2024-04-26T22:28:03.915035+0000","last_fullsized":"2024-04-26T22:28:03.915035+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:12.857637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093248+0000","last_change":"2024-04-26T22:28:01.558972+0000","last_active":"2024-04-26T22:28:04.093248+0000","last_peered":"2024-04-26T22:28:04.093248+0000","last_clean":"2024-04-26T22:28:04.093248+0000","last_became_active":"2024-04-26T22:28:01.558693+0000","last_became_peered":"2024-04-26T22:28:01.558693+0000","last_unstale":"2024-04-26T22:28:04.093248+0000","last_undegraded":"2024-04-26T22:28:04.093248+0000","last_fullsized":"2024-04-26T22:28:04.093248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:03.521303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.914721+0000","last_change":"2024-04-26T22:28:02.562354+0000","last_active":"2024-04-26T22:28:03.914721+0000","last_peered":"2024-04-26T22:28:03.914721+0000","last_clean":"2024-04-26T22:28:03.914721+0000","last_became_active":"2024-04-26T22:28:02.562094+0000","last_became_peered":"2024-04-26T22:28:02.562094+0000","last_unstale":"2024-04-26T22:28:03.914721+0000","last_undegraded":"2024-04-26T22:28:03.914721+0000","last_fullsized":"2024-04-26T22:28:03.914721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:26:10.313571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560258+0000","last_change":"2024-04-26T22:28:01.551281+0000","last_active":"2024-04-26T22:28:03.560258+0000","last_peered":"2024-04-26T22:28:03.560258+0000","last_clean":"2024-04-26T22:28:03.560258+0000","last_became_active":"2024-04-26T22:28:01.550195+0000","last_became_peered":"2024-04-26T22:28:01.550195+0000","last_unstale":"2024-04-26T22:28:03.560258+0000","last_undegraded":"2024-04-26T22:28:03.560258+0000","last_fullsized":"2024-04-26T22:28:03.560258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:18:36.882508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141785+0000","last_change":"2024-04-26T22:28:02.556199+0000","last_active":"2024-04-26T22:28:04.141785+0000","last_peered":"2024-04-26T22:28:04.141785+0000","last_clean":"2024-04-26T22:28:04.141785+0000","last_became_active":"2024-04-26T22:28:02.555904+0000","last_became_peered":"2024-04-26T22:28:02.555904+0000","last_unstale":"2024-04-26T22:28:04.141785+0000","last_undegraded":"2024-04-26T22:28:04.141785+0000","last_fullsized":"2024-04-26T22:28:04.141785+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:25.234424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559469+0000","last_change":"2024-04-26T22:28:01.553831+0000","last_active":"2024-04-26T22:28:03.559469+0000","last_peered":"2024-04-26T22:28:03.559469+0000","last_clean":"2024-04-26T22:28:03.559469+0000","last_became_active":"2024-04-26T22:28:01.553563+0000","last_became_peered":"2024-04-26T22:28:01.553563+0000","last_unstale":"2024-04-26T22:28:03.559469+0000","last_undegraded":"2024-04-26T22:28:03.559469+0000","last_fullsized":"2024-04-26T22:28:03.559469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.915107+0000","last_change":"2024-04-26T22:28:02.557112+0000","last_active":"2024-04-26T22:28:03.915107+0000","last_peered":"2024-04-26T22:28:03.915107+0000","last_clean":"2024-04-26T22:28:03.915107+0000","last_became_active":"2024-04-26T22:28:02.556819+0000","last_became_peered":"2024-04-26T22:28:02.556819+0000","last_unstale":"2024-04-26T22:28:03.915107+0000","last_undegraded":"2024-04-26T22:28:03.915107+0000","last_fullsized":"2024-04-26T22:28:03.915107+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559687+0000","last_change":"2024-04-26T22:28:01.558893+0000","last_active":"2024-04-26T22:28:03.559687+0000","last_peered":"2024-04-26T22:28:03.559687+0000","last_clean":"2024-04-26T22:28:03.559687+0000","last_became_active":"2024-04-26T22:28:01.558541+0000","last_became_peered":"2024-04-26T22:28:01.558541+0000","last_unstale":"2024-04-26T22:28:03.559687+0000","last_undegraded":"2024-04-26T22:28:03.559687+0000","last_fullsized":"2024-04-26T22:28:03.559687+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.561112+0000","last_change":"2024-04-26T22:28:02.569686+0000","last_active":"2024-04-26T22:28:03.561112+0000","last_peered":"2024-04-26T22:28:03.561112+0000","last_clean":"2024-04-26T22:28:03.561112+0000","last_became_active":"2024-04-26T22:28:02.568741+0000","last_became_peered":"2024-04-26T22:28:02.568741+0000","last_unstale":"2024-04-26T22:28:03.561112+0000","last_undegraded":"2024-04-26T22:28:03.561112+0000","last_fullsized":"2024-04-26T22:28:03.561112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554552+0000","last_change":"2024-04-26T22:28:01.560436+0000","last_active":"2024-04-26T22:28:03.554552+0000","last_peered":"2024-04-26T22:28:03.554552+0000","last_clean":"2024-04-26T22:28:03.554552+0000","last_became_active":"2024-04-26T22:28:01.560153+0000","last_became_peered":"2024-04-26T22:28:01.560153+0000","last_unstale":"2024-04-26T22:28:03.554552+0000","last_undegraded":"2024-04-26T22:28:03.554552+0000","last_fullsized":"2024-04-26T22:28:03.554552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:59.038151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554398+0000","last_change":"2024-04-26T22:28:02.569034+0000","last_active":"2024-04-26T22:28:03.554398+0000","last_peered":"2024-04-26T22:28:03.554398+0000","last_clean":"2024-04-26T22:28:03.554398+0000","last_became_active":"2024-04-26T22:28:02.568666+0000","last_became_peered":"2024-04-26T22:28:02.568666+0000","last_unstale":"2024-04-26T22:28:03.554398+0000","last_undegraded":"2024-04-26T22:28:03.554398+0000","last_fullsized":"2024-04-26T22:28:03.554398+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559560+0000","last_change":"2024-04-26T22:28:01.558609+0000","last_active":"2024-04-26T22:28:03.559560+0000","last_peered":"2024-04-26T22:28:03.559560+0000","last_clean":"2024-04-26T22:28:03.559560+0000","last_became_active":"2024-04-26T22:28:01.558332+0000","last_became_peered":"2024-04-26T22:28:01.558332+0000","last_unstale":"2024-04-26T22:28:03.559560+0000","last_undegraded":"2024-04-26T22:28:03.559560+0000","last_fullsized":"2024-04-26T22:28:03.559560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093237+0000","last_change":"2024-04-26T22:28:02.559178+0000","last_active":"2024-04-26T22:28:04.093237+0000","last_peered":"2024-04-26T22:28:04.093237+0000","last_clean":"2024-04-26T22:28:04.093237+0000","last_became_active":"2024-04-26T22:28:02.558903+0000","last_became_peered":"2024-04-26T22:28:02.558903+0000","last_unstale":"2024-04-26T22:28:04.093237+0000","last_undegraded":"2024-04-26T22:28:04.093237+0000","last_fullsized":"2024-04-26T22:28:04.093237+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:05.325356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554646+0000","last_change":"2024-04-26T22:28:01.551370+0000","last_active":"2024-04-26T22:28:03.554646+0000","last_peered":"2024-04-26T22:28:03.554646+0000","last_clean":"2024-04-26T22:28:03.554646+0000","last_became_active":"2024-04-26T22:28:01.550977+0000","last_became_peered":"2024-04-26T22:28:01.550977+0000","last_unstale":"2024-04-26T22:28:03.554646+0000","last_undegraded":"2024-04-26T22:28:03.554646+0000","last_fullsized":"2024-04-26T22:28:03.554646+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:59.490758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.554440+0000","last_change":"2024-04-26T22:28:02.555664+0000","last_active":"2024-04-26T22:28:03.554440+0000","last_peered":"2024-04-26T22:28:03.554440+0000","last_clean":"2024-04-26T22:28:03.554440+0000","last_became_active":"2024-04-26T22:28:02.555366+0000","last_became_peered":"2024-04-26T22:28:02.555366+0000","last_unstale":"2024-04-26T22:28:03.554440+0000","last_undegraded":"2024-04-26T22:28:03.554440+0000","last_fullsized":"2024-04-26T22:28:03.554440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.559189+0000","last_change":"2024-04-26T22:28:01.555714+0000","last_active":"2024-04-26T22:28:03.559189+0000","last_peered":"2024-04-26T22:28:03.559189+0000","last_clean":"2024-04-26T22:28:03.559189+0000","last_became_active":"2024-04-26T22:28:01.555437+0000","last_became_peered":"2024-04-26T22:28:01.555437+0000","last_unstale":"2024-04-26T22:28:03.559189+0000","last_undegraded":"2024-04-26T22:28:03.559189+0000","last_fullsized":"2024-04-26T22:28:03.559189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:50:30.964071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296056+0000","last_change":"2024-04-26T22:28:02.570475+0000","last_active":"2024-04-26T22:28:04.296056+0000","last_peered":"2024-04-26T22:28:04.296056+0000","last_clean":"2024-04-26T22:28:04.296056+0000","last_became_active":"2024-04-26T22:28:02.570131+0000","last_became_peered":"2024-04-26T22:28:02.570131+0000","last_unstale":"2024-04-26T22:28:04.296056+0000","last_undegraded":"2024-04-26T22:28:04.296056+0000","last_fullsized":"2024-04-26T22:28:04.296056+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:22.616877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560297+0000","last_change":"2024-04-26T22:28:01.549998+0000","last_active":"2024-04-26T22:28:03.560297+0000","last_peered":"2024-04-26T22:28:03.560297+0000","last_clean":"2024-04-26T22:28:03.560297+0000","last_became_active":"2024-04-26T22:28:01.549526+0000","last_became_peered":"2024-04-26T22:28:01.549526+0000","last_unstale":"2024-04-26T22:28:03.560297+0000","last_undegraded":"2024-04-26T22:28:03.560297+0000","last_fullsized":"2024-04-26T22:28:03.560297+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:38:11.850900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:03.560047+0000","last_change":"2024-04-26T22:28:02.565124+0000","last_active":"2024-04-26T22:28:03.560047+0000","last_peered":"2024-04-26T22:28:03.560047+0000","last_clean":"2024-04-26T22:28:03.560047+0000","last_became_active":"2024-04-26T22:28:02.564865+0000","last_became_peered":"2024-04-26T22:28:02.564865+0000","last_unstale":"2024-04-26T22:28:03.560047+0000","last_undegraded":"2024-04-26T22:28:03.560047+0000","last_fullsized":"2024-04-26T22:28:03.560047+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:17.864986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.296172+0000","last_change":"2024-04-26T22:28:01.555358+0000","last_active":"2024-04-26T22:28:04.296172+0000","last_peered":"2024-04-26T22:28:04.296172+0000","last_clean":"2024-04-26T22:28:04.296172+0000","last_became_active":"2024-04-26T22:28:01.555056+0000","last_became_peered":"2024-04-26T22:28:01.555056+0000","last_unstale":"2024-04-26T22:28:04.296172+0000","last_undegraded":"2024-04-26T22:28:04.296172+0000","last_fullsized":"2024-04-26T22:28:04.296172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:07.557474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.141923+0000","last_change":"2024-04-26T22:28:02.562956+0000","last_active":"2024-04-26T22:28:04.141923+0000","last_peered":"2024-04-26T22:28:04.141923+0000","last_clean":"2024-04-26T22:28:04.141923+0000","last_became_active":"2024-04-26T22:28:02.562678+0000","last_became_peered":"2024-04-26T22:28:02.562678+0000","last_unstale":"2024-04-26T22:28:04.141923+0000","last_undegraded":"2024-04-26T22:28:04.141923+0000","last_fullsized":"2024-04-26T22:28:04.141923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:01.529467+0000","last_clean_scrub_stamp":"2024-04-26T22:28:01.529467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:55.930956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.093122+0000","last_change":"2024-04-26T22:28:01.554918+0000","last_active":"2024-04-26T22:28:04.093122+0000","last_peered":"2024-04-26T22:28:04.093122+0000","last_clean":"2024-04-26T22:28:04.093122+0000","last_became_active":"2024-04-26T22:28:01.554637+0000","last_became_peered":"2024-04-26T22:28:01.554637+0000","last_unstale":"2024-04-26T22:28:04.093122+0000","last_undegraded":"2024-04-26T22:28:04.093122+0000","last_fullsized":"2024-04-26T22:28:04.093122+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:00.522730+0000","last_clean_scrub_stamp":"2024-04-26T22:28:00.522730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:35.856931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T22:28:04.142222+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:04.142222+0000","last_peered":"2024-04-26T22:28:04.142222+0000","last_clean":"2024-04-26T22:28:04.142222+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:04.142222+0000","last_undegraded":"2024-04-26T22:28:04.142222+0000","last_fullsized":"2024-04-26T22:28:04.142222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":725149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":135363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":134795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":134761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":725059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:11.390 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:28:11.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T22:28:11.534 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.534+0000 7fa2f5aea640 1 Processor -- start 2024-04-26T22:28:11.535 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.534+0000 7fa2f5aea640 1 -- start start 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2f5aea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2f5aea640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa2f00b2750 0x7fa2f00b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2f5aea640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2f00b2550 con 0x7fa2f015f9c0 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2f5aea640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa2f00d1f70 con 0x7fa2f00b2750 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2f5aea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f00d0310 con 0x7fa2f00cf990 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.535+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:44330/0 (socket says 172.21.15.62:44330) 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 -- 172.21.15.62:0/1284976004 learned_addr learned my addr 172.21.15.62:0/1284976004 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2effff640 1 --2- 172.21.15.62:0/1284976004 >> v2:172.21.15.174:3300/0 conn(0x7fa2f00b2750 0x7fa2f00b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 -- 172.21.15.62:0/1284976004 >> v2:172.21.15.174:3300/0 conn(0x7fa2f00b2750 msgr2=0x7fa2f00b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 --2- 172.21.15.62:0/1284976004 >> v2:172.21.15.174:3300/0 conn(0x7fa2f00b2750 0x7fa2f00b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2ee7fc640 1 -- 172.21.15.62:0/1284976004 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716099286 0 0) 0x7fa2e0001110 con 0x7fa2f015f9c0 2024-04-26T22:28:11.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 -- 172.21.15.62:0/1284976004 >> v1:172.21.15.174:6789/0 conn(0x7fa2f015f9c0 legacy=0x7fa2f00d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.536+0000 7fa2eeffd640 1 -- 172.21.15.62:0/1284976004 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f006dcf0 con 0x7fa2f00cf990 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2eeffd640 1 --2- 172.21.15.62:0/1284976004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be44afb4a5c739ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2ee7fc640 1 -- 172.21.15.62:0/1284976004 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4037070 con 0x7fa2f00cf990 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2ee7fc640 1 -- 172.21.15.62:0/1284976004 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e4032440 con 0x7fa2f00cf990 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2ee7fc640 1 -- 172.21.15.62:0/1284976004 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4032810 con 0x7fa2f00cf990 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 -- 172.21.15.62:0/1284976004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 msgr2=0x7fa2f00cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/1284976004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e402a1c0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 -- 172.21.15.62:0/1284976004 shutdown_connections 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/1284976004 >> v2:172.21.15.174:3300/0 conn(0x7fa2f00b2750 0x7fa2f00b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/1284976004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f00cf990 0x7fa2f00cfdd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 -- 172.21.15.62:0/1284976004 >> 172.21.15.62:0/1284976004 conn(0x7fa2f00d2f80 msgr2=0x7fa2f00d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 -- 172.21.15.62:0/1284976004 shutdown_connections 2024-04-26T22:28:11.538 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.537+0000 7fa2f5aea640 1 -- 172.21.15.62:0/1284976004 wait complete. 2024-04-26T22:28:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 Processor -- start 2024-04-26T22:28:11.539 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 -- start start 2024-04-26T22:28:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 0x7fa2f0157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.541 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 0x7fa2f00c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f006e590 con 0x7fa2f00cf990 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.539+0000 7fa2f5aea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f006e2b0 con 0x7fa2f00b2750 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2f5aea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f006e870 con 0x7fa2f015f9c0 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43422/0 (socket says 172.21.15.62:43422) 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 -- 172.21.15.62:0/2681112597 learned_addr learned my addr 172.21.15.62:0/2681112597 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2effff640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 0x7fa2f00c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2ef7fe640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 0x7fa2f0157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 -- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 msgr2=0x7fa2f00c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 0x7fa2f00c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 -- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 msgr2=0x7fa2f0157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 0x7fa2f0157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.542 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 -- 172.21.15.62:0/2681112597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f01ea650 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2ef7fe640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 0x7fa2f0157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2effff640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 0x7fa2f00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2eeffd640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4037040 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e4004230 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f01ea930 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e403c670 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.540+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f01eaef0 con 0x7fa2f00cf990 2024-04-26T22:28:11.543 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.541+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f006a360 con 0x7fa2f00cf990 2024-04-26T22:28:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.542+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2e403a070 con 0x7fa2f00cf990 2024-04-26T22:28:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.542+0000 7fa2d7fff640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 0x7fa2dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.542+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e40ac2f0 con 0x7fa2f00cf990 2024-04-26T22:28:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.543+0000 7fa2ef7fe640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 0x7fa2dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.544 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.543+0000 7fa2ef7fe640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 0x7fa2dc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000f60 tx=0x7fa2e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.547 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.547+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2e4041020 con 0x7fa2f00cf990 2024-04-26T22:28:11.802 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.802+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa2f015c520 con 0x7fa2f00cf990 2024-04-26T22:28:11.803 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.803+0000 7fa2d7fff640 1 -- 172.21.15.62:0/2681112597 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fa2e407a720 con 0x7fa2f00cf990 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ== 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T22:28:11.804 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 msgr2=0x7fa2dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 0x7fa2dc06edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0000f60 tx=0x7fa2e0009040 comp rx=0 tx=0).stop 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 msgr2=0x7fa2f01e9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009870 tx=0x7fa2e40046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 shutdown_connections 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa2dc06c8f0 0x7fa2dc06edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa2f015f9c0 0x7fa2f00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.806 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa2f00cf990 0x7fa2f01e9b90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 --2- 172.21.15.62:0/2681112597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2f00b2750 0x7fa2f0157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 >> 172.21.15.62:0/2681112597 conn(0x7fa2f00d2f80 msgr2=0x7fa2f015b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 shutdown_connections 2024-04-26T22:28:11.807 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.806+0000 7fa2f5aea640 1 -- 172.21.15.62:0/2681112597 wait complete. 2024-04-26T22:28:11.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:11.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.917+0000 7fabf4d46640 1 Processor -- start 2024-04-26T22:28:11.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.917+0000 7fabf4d46640 1 -- start start 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabf4d46640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabf0162b40 0x7fabf00d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabf4d46640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00d2670 con 0x7fabf00d3c80 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabf4d46640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00d0210 con 0x7fabf00d3820 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabf4d46640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fabf00d04f0 con 0x7fabf0162b40 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabeed9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fabf00d3c80 0x7fabf00d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42372/0 (socket says 172.21.15.62:42372) 2024-04-26T22:28:11.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.918+0000 7fabeed9d640 1 -- 172.21.15.62:0/1971599058 learned_addr learned my addr 172.21.15.62:0/1971599058 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabee59c640 1 --2- 172.21.15.62:0/1971599058 >> v2:172.21.15.174:3300/0 conn(0x7fabf0162b40 0x7fabf00d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185666407 0 0) 0x7fabf00d2670 con 0x7fabf00d3c80 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabdc003520 con 0x7fabf00d3c80 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabee59c640 1 -- 172.21.15.62:0/1971599058 >> v1:172.21.15.174:6789/0 conn(0x7fabf00d3820 legacy=0x7fabf00d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabee59c640 1 -- 172.21.15.62:0/1971599058 >> v1:172.21.15.62:6789/0 conn(0x7fabf00d3c80 legacy=0x7fabf00d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4098055623 0 0) 0x7fabdc003520 con 0x7fabf00d3c80 2024-04-26T22:28:11.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.919+0000 7fabee59c640 1 -- 172.21.15.62:0/1971599058 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf00d07d0 con 0x7fabf0162b40 2024-04-26T22:28:11.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabee59c640 1 --2- 172.21.15.62:0/1971599058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf0162b40 0x7fabf00d5bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009d90 tx=0x7fabe002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9fcb35b52e742f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8001670 con 0x7fabf0162b40 2024-04-26T22:28:11.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0007540 con 0x7fabf0162b40 2024-04-26T22:28:11.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabed59a640 1 -- 172.21.15.62:0/1971599058 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe00078f0 con 0x7fabf0162b40 2024-04-26T22:28:11.921 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabf4d46640 1 -- 172.21.15.62:0/1971599058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf0162b40 msgr2=0x7fabf00d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.920+0000 7fabf4d46640 1 --2- 172.21.15.62:0/1971599058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf0162b40 0x7fabf00d5bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009d90 tx=0x7fabe002a710 comp rx=0 tx=0).stop 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.921+0000 7fabf4d46640 1 -- 172.21.15.62:0/1971599058 shutdown_connections 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.921+0000 7fabf4d46640 1 --2- 172.21.15.62:0/1971599058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf0162b40 0x7fabf00d5bd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.921+0000 7fabf4d46640 1 -- 172.21.15.62:0/1971599058 >> 172.21.15.62:0/1971599058 conn(0x7fabf009e550 msgr2=0x7fabf00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.921+0000 7fabf4d46640 1 -- 172.21.15.62:0/1971599058 shutdown_connections 2024-04-26T22:28:11.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.921+0000 7fabf4d46640 1 -- 172.21.15.62:0/1971599058 wait complete. 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.922+0000 7fabf4d46640 1 Processor -- start 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.922+0000 7fabf4d46640 1 -- start start 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf00d3820 0x7fabf015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabf00d3c80 0x7fabf0154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.924 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00d4110 con 0x7fabf0162b40 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00d0d90 con 0x7fabf00d3820 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabf4d46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabf00b5bb0 con 0x7fabf00d3c80 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43430/0 (socket says 172.21.15.62:43430) 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 -- 172.21.15.62:0/2036465962 learned_addr learned my addr 172.21.15.62:0/2036465962 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabee59c640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf00d3820 0x7fabf015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabedd9b640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabf00d3c80 0x7fabf0154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 -- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabf00d3c80 msgr2=0x7fabf0154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabf00d3c80 0x7fabf0154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 -- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf00d3820 msgr2=0x7fabf015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf00d3820 0x7fabf015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 -- 172.21.15.62:0/2036465962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf0154b80 con 0x7fabf0162b40 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.923+0000 7fabeed9d640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c750 tx=0x7fabe800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.924+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8017070 con 0x7fabf0162b40 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.924+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe8005ad0 con 0x7fabf0162b40 2024-04-26T22:28:11.925 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.924+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf00ceff0 con 0x7fabf0162b40 2024-04-26T22:28:11.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.924+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8007070 con 0x7fabf0162b40 2024-04-26T22:28:11.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.924+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf00cc3b0 con 0x7fabf0162b40 2024-04-26T22:28:11.926 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.925+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf006a360 con 0x7fabf0162b40 2024-04-26T22:28:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.925+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe801a070 con 0x7fabf0162b40 2024-04-26T22:28:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.926+0000 7fabe6ffd640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 0x7fabd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.926+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fabe808ae50 con 0x7fabf0162b40 2024-04-26T22:28:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.926+0000 7fabee59c640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 0x7fabd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:11.927 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.926+0000 7fabee59c640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 0x7fabd006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabe0034000 tx=0x7fabe002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:11.930 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:11.930+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabe8058c30 con 0x7fabf0162b40 2024-04-26T22:28:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.152+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf00d2960 con 0x7fabf0162b40 2024-04-26T22:28:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.152+0000 7fabe6ffd640 1 -- 172.21.15.62:0/2036465962 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5184 (secure 0 0 0) 0x7fabe805cf30 con 0x7fabf0162b40 2024-04-26T22:28:12.153 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:12.153 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:12.153 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 16 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 msgr2=0x7fabd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 0x7fabd006edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fabe0034000 tx=0x7fabe002f000 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 msgr2=0x7fabf00c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c750 tx=0x7fabe800cc10 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 shutdown_connections 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fabd006c8f0 0x7fabd006edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fabf0162b40 0x7fabf00c92e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabf00d3c80 0x7fabf0154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 --2- 172.21.15.62:0/2036465962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf00d3820 0x7fabf015f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.155 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.154+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 >> 172.21.15.62:0/2036465962 conn(0x7fabf009e550 msgr2=0x7fabf006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.155+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 shutdown_connections 2024-04-26T22:28:12.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.155+0000 7fabf4d46640 1 -- 172.21.15.62:0/2036465962 wait complete. 2024-04-26T22:28:12.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T22:28:01.910880+0000', 'modified': '2024-04-26T22:28:04.932395+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4837}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4837': {'gid': 4837, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.62:6834/710033080', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6832', 'nonce': 710033080}, {'type': 'v1', 'addr': '172.21.15.62:6834', 'nonce': 710033080}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4837, 'qdb_cluster': [4837]} 2024-04-26T22:28:12.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:28:12.569 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T22:28:12.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.673+0000 7fbc76cd1640 1 Processor -- start 2024-04-26T22:28:12.673 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.673+0000 7fbc76cd1640 1 -- start start 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc76cd1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbc70160a00 0x7fbc700b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc76cd1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc700cf740 con 0x7fbc700d0540 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc76cd1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc700cf080 con 0x7fbc700cf9c0 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc76cd1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fbc700ce7e0 con 0x7fbc70160a00 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc7526e640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbc700d0540 0x7fbc700b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:42376/0 (socket says 172.21.15.62:42376) 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc7526e640 1 -- 172.21.15.62:0/1109534732 learned_addr learned my addr 172.21.15.62:0/1109534732 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/1109534732 >> v2:172.21.15.174:3300/0 conn(0x7fbc70160a00 0x7fbc700b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304088342 0 0) 0x7fbc700cf740 con 0x7fbc700d0540 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.674+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc5c003520 con 0x7fbc700d0540 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc74a6d640 1 -- 172.21.15.62:0/1109534732 >> v1:172.21.15.174:6789/0 conn(0x7fbc700cf9c0 legacy=0x7fbc700cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.675 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961789161 0 0) 0x7fbc700cf080 con 0x7fbc700cf9c0 2024-04-26T22:28:12.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc74a6d640 1 -- 172.21.15.62:0/1109534732 >> v1:172.21.15.62:6789/0 conn(0x7fbc700d0540 legacy=0x7fbc700b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.676 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc74a6d640 1 -- 172.21.15.62:0/1109534732 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc700ceac0 con 0x7fbc70160a00 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/1109534732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700b1f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e49c0a3b6e5b2455 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.675+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037070 con 0x7fbc70160a00 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc68032640 con 0x7fbc70160a00 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc6f7fe640 1 -- 172.21.15.62:0/1109534732 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68032a10 con 0x7fbc70160a00 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1109534732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 msgr2=0x7fbc700b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1109534732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700b1f90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbc680097a0 tx=0x7fbc6802a2c0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1109534732 shutdown_connections 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1109534732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700b1f90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1109534732 >> 172.21.15.62:0/1109534732 conn(0x7fbc7009ba50 msgr2=0x7fbc7009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1109534732 shutdown_connections 2024-04-26T22:28:12.677 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.676+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1109534732 wait complete. 2024-04-26T22:28:12.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.677+0000 7fbc76cd1640 1 Processor -- start 2024-04-26T22:28:12.679 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.677+0000 7fbc76cd1640 1 -- start start 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbc700cf9c0 0x7fbc7015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc700b2690 con 0x7fbc700d0540 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc70160de0 con 0x7fbc70160a00 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc76cd1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbc700cc230 con 0x7fbc700cf9c0 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43432/0 (socket says 172.21.15.62:43432) 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc6ffff640 1 -- 172.21.15.62:0/1844671167 learned_addr learned my addr 172.21.15.62:0/1844671167 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbc700cf9c0 0x7fbc7015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.678+0000 7fbc7526e640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 -- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbc700cf9c0 msgr2=0x7fbc7015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbc700cf9c0 0x7fbc7015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.680 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 -- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 msgr2=0x7fbc700cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 -- 172.21.15.62:0/1844671167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc700cc510 con 0x7fbc700d0540 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000c8c0 tx=0x7fbc6000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6d7fa640 1 -- 172.21.15.62:0/1844671167 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60017070 con 0x7fbc700d0540 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6d7fa640 1 -- 172.21.15.62:0/1844671167 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc60003dc0 con 0x7fbc700d0540 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc700cc7f0 con 0x7fbc700d0540 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc6d7fa640 1 -- 172.21.15.62:0/1844671167 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600052b0 con 0x7fbc700d0540 2024-04-26T22:28:12.681 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.679+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc700ccdb0 con 0x7fbc700d0540 2024-04-26T22:28:12.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.680+0000 7fbc6d7fa640 1 -- 172.21.15.62:0/1844671167 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc6001a070 con 0x7fbc700d0540 2024-04-26T22:28:12.683 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.681+0000 7fbc6d7fa640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 0x7fbc5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.681+0000 7fbc6d7fa640 1 -- 172.21.15.62:0/1844671167 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbc6008b1e0 con 0x7fbc700d0540 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.681+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 0x7fbc5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.682+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 0x7fbc5006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc68034000 tx=0x7fbc6802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.682+0000 7fbc799ec740 1 Processor -- start 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.682+0000 7fbc799ec740 1 -- start start 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f953700230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f95361c740 con 0x55f9536b8120 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9535b83a0 con 0x55f9536ffce0 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55f953618700 con 0x55f9536e2440 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.684 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43436/0 (socket says 172.21.15.62:43436) 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 -- 172.21.15.62:0/2778281161 learned_addr learned my addr 172.21.15.62:0/2778281161 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f953700230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 -- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 msgr2=0x55f9536e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 -- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 msgr2=0x55f953700230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f953700230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 -- 172.21.15.62:0/2778281161 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9535d4750 con 0x55f9536b8120 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc7526e640 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbc58001590 tx=0x7fbc58001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fe4a6ba99a3e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc667fc640 1 -- 172.21.15.62:0/2778281161 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58019070 con 0x55f9536b8120 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc667fc640 1 -- 172.21.15.62:0/2778281161 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc58005850 con 0x55f9536b8120 2024-04-26T22:28:12.685 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.683+0000 7fbc667fc640 1 -- 172.21.15.62:0/2778281161 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58005c00 con 0x55f9536b8120 2024-04-26T22:28:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 -- 172.21.15.62:0/2778281161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 msgr2=0x55f9536fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.689 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbc58001590 tx=0x7fbc58001a50 comp rx=0 tx=0).stop 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 -- 172.21.15.62:0/2778281161 shutdown_connections 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536b8120 0x55f9536fe3a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f953700230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 --2- 172.21.15.62:0/2778281161 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 -- 172.21.15.62:0/2778281161 >> 172.21.15.62:0/2778281161 conn(0x55f9536b4a50 msgr2=0x55f9536fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 -- 172.21.15.62:0/2778281161 shutdown_connections 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.684+0000 7fbc799ec740 1 -- 172.21.15.62:0/2778281161 wait complete. 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.685+0000 7fbc799ec740 1 -- start start 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.685+0000 7fbc799ec740 1 Processor -- start 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f9536deb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536f7030 0x55f9536e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9536b8ec0 con 0x55f9536e2440 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9535a3fe0 con 0x55f9536ffce0 2024-04-26T22:28:12.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55f9536b2980 con 0x55f9536f7030 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 10 client.0 _ll_register_callbacks cb 0x55f9535ba150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:43442/0 (socket says 172.21.15.62:43442) 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 -- 172.21.15.62:0/3160486372 learned_addr learned my addr 172.21.15.62:0/3160486372 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f9536deb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc7526e640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536f7030 0x55f9536e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 -- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536f7030 msgr2=0x55f9536e43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536f7030 0x55f9536e43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 -- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 msgr2=0x55f9536deb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f9536deb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 -- 172.21.15.62:0/3160486372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f95332e2d0 con 0x55f9536e2440 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009560 tx=0x7fbc68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4637bd13d1def0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:28:12.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68037040 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbc6803e9b0 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.686+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc58005850 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fbc68056050 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 10 client.5265 fetch_fsmap learned FSMap version 16 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f9533f86b0 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc58005850 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fbc68059020 con 0x55f9536e2440 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 10 client.5265 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 10 client.5265 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 20 client.5265 populate_metadata read hostname 'smithi062' 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f95370d6b0 0x55f95370da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 4 client.5265 mds_command: new command op to 4837 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 --> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f95362a350 con 0x55f95370d6b0 2024-04-26T22:28:12.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f95370d6b0 0x55f95370da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f95370d6b0 0x55f95370da90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389c16203c7519c server_cookie=b84bda9416c9b503 in_seq=0 out_seq=0 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.687+0000 7fbc4f7fe640 10 client.5265 ms_handle_connect on v2:172.21.15.62:6832/710033080 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/3160486372 <== mds.0 v2:172.21.15.62:6832/710033080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f95362a350 con 0x55f95370d6b0 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc4f7fe640 10 client.5265 handle_command_reply: tid=0 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 client.5265 shutdown 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 msgr2=0x55f9536f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbc68009560 tx=0x7fbc68004730 comp rx=0 tx=0).stop 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 shutdown_connections 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f95370d6b0 0x55f95370da90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536f7030 0x55f9536e43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f9536deb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 --2- 172.21.15.62:0/3160486372 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9536f5780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc4f7fe640 0 client.5265 ms_handle_reset on v2:172.21.15.62:6832/710033080 2024-04-26T22:28:12.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 >> 172.21.15.62:0/3160486372 conn(0x55f9536b8120 msgr2=0x55f9536e14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 shutdown_connections 2024-04-26T22:28:12.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 1 -- 172.21.15.62:0/3160486372 wait complete. 2024-04-26T22:28:12.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.688+0000 7fbc799ec740 20 client.5265 trim_cache size 0 max 16384 2024-04-26T22:28:12.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 Processor -- start 2024-04-26T22:28:12.707 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 -- start start 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f95370cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc701fe2e0 con 0x55f9536f7030 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc700c7670 con 0x55f9536ffce0 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbc701f2830 con 0x55f9536e2440 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38390/0 (socket says 172.21.15.62:38390) 2024-04-26T22:28:12.709 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.707+0000 7fbc6ffff640 1 -- 172.21.15.62:0/255194900 learned_addr learned my addr 172.21.15.62:0/255194900 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc7526e640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f95370cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 -- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 msgr2=0x55f9536e4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 -- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 msgr2=0x55f95370cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f95370cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 -- 172.21.15.62:0/255194900 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9536b32a0 con 0x55f9536f7030 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc7526e640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f95370cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc6008b1b0 tx=0x7fbc60089670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75d4f03206a18bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc65ffb640 1 -- 172.21.15.62:0/255194900 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60089430 con 0x55f9536f7030 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc65ffb640 1 -- 172.21.15.62:0/255194900 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc600572a0 con 0x55f9536f7030 2024-04-26T22:28:12.710 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.708+0000 7fbc65ffb640 1 -- 172.21.15.62:0/255194900 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc600616f0 con 0x55f9536f7030 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- 172.21.15.62:0/255194900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 msgr2=0x55f9536fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc6008b1b0 tx=0x7fbc60089670 comp rx=0 tx=0).stop 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- 172.21.15.62:0/255194900 shutdown_connections 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536ffce0 0x55f95370cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536f7030 0x55f9536fd490 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 --2- 172.21.15.62:0/255194900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536e2440 0x55f9536e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- 172.21.15.62:0/255194900 >> 172.21.15.62:0/255194900 conn(0x55f9536b8120 msgr2=0x55f9536f5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- 172.21.15.62:0/255194900 shutdown_connections 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- 172.21.15.62:0/255194900 wait complete. 2024-04-26T22:28:12.711 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 -- start start 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.709+0000 7fbc799ec740 1 Processor -- start 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 0x55f95370faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536ffce0 0x55f95370ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f9536f74b0 con 0x55f9536e2440 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9536f4f80 con 0x55f9536f7030 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55f953710ba0 con 0x55f9536ffce0 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc799ec740 10 client.0 _ll_register_callbacks cb 0x7fbc68035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc74a6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc74a6d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38400/0 (socket says 172.21.15.62:38400) 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.710+0000 7fbc74a6d640 1 -- 172.21.15.62:0/4040327205 learned_addr learned my addr 172.21.15.62:0/4040327205 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 0x55f95370faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc7526e640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536ffce0 0x55f95370ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 -- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536ffce0 msgr2=0x55f95370ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536ffce0 0x55f95370ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 -- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 msgr2=0x55f95370faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 0x55f95370faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 -- 172.21.15.62:0/4040327205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9536e32a0 con 0x55f9536e2440 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 0x55f95370faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc74a6d640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbc68032a40 tx=0x7fbc68009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a70fbc7158151f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6805e020 con 0x55f9536e2440 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbc6803ccf0 con 0x55f9536e2440 2024-04-26T22:28:12.715 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc600572a0 con 0x55f9536e2440 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fbc68068050 con 0x55f9536e2440 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc799ec740 10 client.5271 fetch_fsmap learned FSMap version 16 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.711+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55f953720d10 con 0x55f9536e2440 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc600572a0 con 0x55f9536e2440 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fbc6806b020 con 0x55f9536e2440 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 10 client.5271 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 10 client.5271 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 20 client.5271 populate_metadata read hostname 'smithi062' 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f953720900 0x55f9536e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 4 client.5271 mds_command: new command op to 4837 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.713+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 --> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] -- command(tid 0: {"prefix": "status"}) -- 0x55f9536241c0 con 0x55f953720900 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f953720900 0x55f9536e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc4f7fe640 10 client.5271 ms_handle_connect on v2:172.21.15.62:6832/710033080 2024-04-26T22:28:12.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc6ffff640 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f953720900 0x55f9536e0960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba7d59c00d3432c server_cookie=3d13c41a4511011c in_seq=0 out_seq=0 2024-04-26T22:28:12.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc4f7fe640 1 -- 172.21.15.62:0/4040327205 <== mds.0 v2:172.21.15.62:6832/710033080 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55f9536241c0 con 0x55f953720900 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc4f7fe640 10 client.5271 handle_command_reply: tid=0 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 client.5271 shutdown 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 msgr2=0x55f9537645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbc68032a40 tx=0x7fbc68009190 comp rx=0 tx=0).stop 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 shutdown_connections 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:6832/710033080,v1:172.21.15.62:6834/710033080] conn(0x55f953720900 0x55f9536e0960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f9536ffce0 0x55f95370ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9536f7030 0x55f95370faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.714+0000 7fbc4f7fe640 0 client.5271 ms_handle_reset on v2:172.21.15.62:6832/710033080 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc799ec740 1 --2- 172.21.15.62:0/4040327205 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f9536e2440 0x55f9537645d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 >> 172.21.15.62:0/4040327205 conn(0x55f9536b8120 msgr2=0x55f9536e1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 shutdown_connections 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc799ec740 1 -- 172.21.15.62:0/4040327205 wait complete. 2024-04-26T22:28:12.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc799ec740 20 client.5271 trim_cache size 0 max 16384 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "f70b0a7c-519a-45f1-aa13-a1f2de1080a4", 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 4837, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 10.164542604999999, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 16, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 31, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 31, 2024-04-26T22:28:12.719 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 17.223589128 2024-04-26T22:28:12.720 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-26T22:28:12.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 msgr2=0x7fbc5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 0x7fbc5006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc68034000 tx=0x7fbc6802e000 comp rx=0 tx=0).stop 2024-04-26T22:28:12.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 msgr2=0x7fbc700c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000c8c0 tx=0x7fbc6000cd80 comp rx=0 tx=0).stop 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 shutdown_connections 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbc5006c8f0 0x7fbc5006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc70160a00 0x7fbc700cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc700d0540 0x7fbc700c8a60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 --2- 172.21.15.62:0/1844671167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbc700cf9c0 0x7fbc7015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 >> 172.21.15.62:0/1844671167 conn(0x7fbc7009ba50 msgr2=0x7fbc700aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 shutdown_connections 2024-04-26T22:28:12.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.715+0000 7fbc76cd1640 1 -- 172.21.15.62:0/1844671167 wait complete. 2024-04-26T22:28:12.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:28:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.827+0000 7fc5cc27b640 1 Processor -- start 2024-04-26T22:28:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.827+0000 7fc5cc27b640 1 -- start start 2024-04-26T22:28:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c406fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.829 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 0x7fc5c406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc5c40b1480 0x7fc5c40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c406cb60 con 0x7fc5c40b1480 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c40b59b0 con 0x7fc5c415f4f0 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5cc27b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c4070350 con 0x7fc5c406c240 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc5c40b1480 0x7fc5c40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc5c40b1480 0x7fc5c40ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38416/0 (socket says 172.21.15.62:38416) 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 -- 172.21.15.62:0/3782462105 learned_addr learned my addr 172.21.15.62:0/3782462105 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5c9816640 1 --2- 172.21.15.62:0/3782462105 >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 0x7fc5c406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca017640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c406fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 -- 172.21.15.62:0/3782462105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 msgr2=0x7fc5c406fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c406fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 -- 172.21.15.62:0/3782462105 >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 msgr2=0x7fc5c406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 --2- 172.21.15.62:0/3782462105 >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 0x7fc5c406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5ca818640 1 -- 172.21.15.62:0/3782462105 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c40b5c70 con 0x7fc5c40b1480 2024-04-26T22:28:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.828+0000 7fc5c9816640 1 --2- 172.21.15.62:0/3782462105 >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 0x7fc5c406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.829+0000 7fc5ca818640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c40b1480 0x7fc5c40ae8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b4950 tx=0x7fc5bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8195c7231a61aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.829+0000 7fc5c9015640 1 -- 172.21.15.62:0/3782462105 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc036070 con 0x7fc5c40b1480 2024-04-26T22:28:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.829+0000 7fc5c9015640 1 -- 172.21.15.62:0/3782462105 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc031600 con 0x7fc5c40b1480 2024-04-26T22:28:12.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.829+0000 7fc5c9015640 1 -- 172.21.15.62:0/3782462105 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0319d0 con 0x7fc5c40b1480 2024-04-26T22:28:12.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 -- 172.21.15.62:0/3782462105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c40b1480 msgr2=0x7fc5c40ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c40b1480 0x7fc5c40ae8c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40b4950 tx=0x7fc5bc02a6a0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 -- 172.21.15.62:0/3782462105 shutdown_connections 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c40b1480 0x7fc5c40ae8c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/3782462105 >> v2:172.21.15.174:3300/0 conn(0x7fc5c415f4f0 0x7fc5c406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/3782462105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c406fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 -- 172.21.15.62:0/3782462105 >> 172.21.15.62:0/3782462105 conn(0x7fc5c40b3b70 msgr2=0x7fc5c40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 -- 172.21.15.62:0/3782462105 shutdown_connections 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.830+0000 7fc5cc27b640 1 -- 172.21.15.62:0/3782462105 wait complete. 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.831+0000 7fc5cc27b640 1 Processor -- start 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.831+0000 7fc5cc27b640 1 -- start start 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c41f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5c40b1480 0x7fc5c41e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40aee00 con 0x7fc5c415f4f0 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40b6230 con 0x7fc5c40b1480 2024-04-26T22:28:12.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c40b0fc0 con 0x7fc5c406c240 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38432/0 (socket says 172.21.15.62:38432) 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 -- 172.21.15.62:0/794339354 learned_addr learned my addr 172.21.15.62:0/794339354 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca017640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c41f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5c9816640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5c40b1480 0x7fc5c41e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 -- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 msgr2=0x7fc5c41f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c41f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 -- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5c40b1480 msgr2=0x7fc5c41e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5c40b1480 0x7fc5c41e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 -- 172.21.15.62:0/794339354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c41e5aa0 con 0x7fc5c415f4f0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5ca818640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009f40 tx=0x7fc5bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc041020 con 0x7fc5c415f4f0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0077a0 con 0x7fc5c415f4f0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c41e5d80 con 0x7fc5c415f4f0 2024-04-26T22:28:12.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.832+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc036040 con 0x7fc5c415f4f0 2024-04-26T22:28:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.833+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41e6340 con 0x7fc5c415f4f0 2024-04-26T22:28:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.834+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c40b4bb0 con 0x7fc5c415f4f0 2024-04-26T22:28:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.834+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5bc044020 con 0x7fc5c415f4f0 2024-04-26T22:28:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.834+0000 7fc5baffd640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:12.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.835+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fc5bc0ac730 con 0x7fc5c415f4f0 2024-04-26T22:28:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.835+0000 7fc5ca017640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:12.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.835+0000 7fc5ca017640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0006fd0 tx=0x7fc5c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:12.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:12.839+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5c40b4bb0 con 0x7fc5c415f4f0 2024-04-26T22:28:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.062+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc5c40d2320 con 0x7fc5c415f4f0 2024-04-26T22:28:13.063 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.063+0000 7fc5baffd640 1 -- 172.21.15.62:0/794339354 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fc5bc04b030 con 0x7fc5c415f4f0 2024-04-26T22:28:13.063 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 msgr2=0x7fc5a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 0x7fc5a806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0006fd0 tx=0x7fc5c0008040 comp rx=0 tx=0).stop 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 msgr2=0x7fc5c40b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009f40 tx=0x7fc5bc004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 shutdown_connections 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc5a806c8f0 0x7fc5a806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc5c415f4f0 0x7fc5c40b2b30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:13.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5c40b1480 0x7fc5c41e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 --2- 172.21.15.62:0/794339354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc5c406c240 0x7fc5c41f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 >> 172.21.15.62:0/794339354 conn(0x7fc5c40b3b70 msgr2=0x7fc5c40b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 shutdown_connections 2024-04-26T22:28:13.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.065+0000 7fc5cc27b640 1 -- 172.21.15.62:0/794339354 wait complete. 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:28:13.076 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:13.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.076 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-26T22:28:13.093 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6c94:f27c:3545:34e3/64 scope link noprefixroute 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 46:95:32:b9:0b:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2a:a6ff:fef2:1980/64 scope link 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:13.094 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d2:58:c7:f5:1b:26 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d058:c7ff:fef5:1b26/64 scope link 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 46:95:32:b9:0b:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::4495:32ff:feb9:bf0/64 scope link 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:13.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.095 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:28:13.156 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-26T22:28:13.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.158 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-26T22:28:13.214 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-26T22:28:13.214 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:28:13.214 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:28:13.214 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:28:13.214 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:28:13.215 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:28:13.302 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:28:13.333 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:13.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.340 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:28:13.355 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T22:28:13.355 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:28:13.355 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:13.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-26T22:28:13.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:13.419 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:13.474 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:13.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:13.475 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:13.529 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:28:13.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:13.612 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:13.612 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:13.612 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:13.621 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:13.626 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:13.626 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:13.626 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:13.626 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:28:13.627 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:28:13.628 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:28:13.628 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:28:13.628 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:28:13.629 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:28:13.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:13.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:13.695 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:28:13.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:28:13.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:28:13.787 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:28:13.791 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:13.797 INFO:teuthology.orchestra.run.smithi062.stdout:client4987 2024-04-26T22:28:13.797 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:13.797 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:13.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:28:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.995+0000 7f0b29178640 1 Processor -- start 2024-04-26T22:28:13.995 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.995+0000 7f0b29178640 1 -- start start 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b29178640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b240d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b29178640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240d3220 con 0x7f0b2415f400 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b29178640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b240d5ef0 con 0x7f0b240d3c80 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b29178640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240d1290 con 0x7f0b240d2180 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b22d9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0b2415f400 0x7f0b240d0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53476/0 (socket says 172.21.15.62:53476) 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b22d9d640 1 -- 172.21.15.62:0/1769515877 learned_addr learned my addr 172.21.15.62:0/1769515877 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47106122 0 0) 0x7f0b240d3220 con 0x7f0b2415f400 2024-04-26T22:28:13.999 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b2259c640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b240d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.996+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b10003520 con 0x7f0b2415f400 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1025351082 0 0) 0x7f0b10003520 con 0x7f0b2415f400 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 msgr2=0x7f0b240d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b240d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 >> v1:172.21.15.174:6789/0 conn(0x7f0b240d3c80 legacy=0x7f0b240b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240d00a0 con 0x7f0b2415f400 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f0b1c001670 con 0x7f0b2415f400 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 >> v1:172.21.15.62:6789/0 conn(0x7f0b2415f400 legacy=0x7f0b240d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b10003f60 0x7f0b10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b10004d40 0x7f0b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d1290 con 0x7f0b10004d40 2024-04-26T22:28:14.000 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1000baa0 con 0x7f0b10003f60 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1000bca0 con 0x7f0b240d2180 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b10004d40 0x7f0b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2259c640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b10003f60 0x7f0b10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b22d9d640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 -- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 msgr2=0x7f0b100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b100039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 -- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b10003f60 msgr2=0x7f0b10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b10003f60 0x7f0b10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b2359e640 1 -- 172.21.15.62:0/1769515877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10003520 con 0x7f0b10004d40 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.997+0000 7f0b22d9d640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b240d2180 0x7f0b100039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.998+0000 7f0b2359e640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b10004d40 0x7f0b100051c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0b14001e50 tx=0x7f0b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9dee4380972fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:14.001 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.998+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14001a70 con 0x7f0b10004d40 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.998+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b14003970 con 0x7f0b10004d40 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.998+0000 7f0b21d9b640 1 -- 172.21.15.62:0/1769515877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14003d60 con 0x7f0b10004d40 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 -- 172.21.15.62:0/1769515877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b10004d40 msgr2=0x7f0b100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 --2- 172.21.15.62:0/1769515877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b10004d40 0x7f0b100051c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0b14001e50 tx=0x7f0b10005830 comp rx=0 tx=0).stop 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b2359e640 1 -- 172.21.15.62:0/1769515877 reap_dead start 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 -- 172.21.15.62:0/1769515877 shutdown_connections 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 -- 172.21.15.62:0/1769515877 >> 172.21.15.62:0/1769515877 conn(0x7f0b2409e510 msgr2=0x7f0b240d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 -- 172.21.15.62:0/1769515877 shutdown_connections 2024-04-26T22:28:14.002 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:13.999+0000 7f0b29178640 1 -- 172.21.15.62:0/1769515877 wait complete. 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.000+0000 7f0b29178640 1 Processor -- start 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.000+0000 7f0b29178640 1 -- start start 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b10004d40 0x7f0b240ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b240c96d0 0x7f0b240c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d0660 con 0x7f0b2409fab0 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d40b0 con 0x7f0b240c96d0 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b29178640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240cfd50 con 0x7f0b10004d40 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38456/0 (socket says 172.21.15.62:38456) 2024-04-26T22:28:14.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 -- 172.21.15.62:0/76733787 learned_addr learned my addr 172.21.15.62:0/76733787 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b22d9d640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b10004d40 0x7f0b240ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2359e640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b240c96d0 0x7f0b240c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 -- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b10004d40 msgr2=0x7f0b240ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b10004d40 0x7f0b240ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 -- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b240c96d0 msgr2=0x7f0b240c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b240c96d0 0x7f0b240c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.001+0000 7f0b2259c640 1 -- 172.21.15.62:0/76733787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2415eb40 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b2259c640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c011940 tx=0x7f0b0c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00faf0 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00fc90 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b240cd5d0 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c014440 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.002+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b240c8bf0 con 0x7f0b2409fab0 2024-04-26T22:28:14.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.003+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b240693e0 con 0x7f0b2409fab0 2024-04-26T22:28:14.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.003+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b0c012040 con 0x7f0b2409fab0 2024-04-26T22:28:14.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.004+0000 7f0b1b7fe640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 0x7f0b0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:14.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.004+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0b0c08c100 con 0x7f0b2409fab0 2024-04-26T22:28:14.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.004+0000 7f0b22d9d640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 0x7f0b0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:14.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.004+0000 7f0b22d9d640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 0x7f0b0406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000ff0 tx=0x7f0b1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:14.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.008+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b0c019030 con 0x7f0b2409fab0 2024-04-26T22:28:14.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.231+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0b240008d0 con 0x7f0b2409fab0 2024-04-26T22:28:14.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.231+0000 7f0b1b7fe640 1 -- 172.21.15.62:0/76733787 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f0b0c05a5c0 con 0x7f0b2409fab0 2024-04-26T22:28:14.231 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 msgr2=0x7f0b0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 0x7f0b0406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000ff0 tx=0x7f0b1c009040 comp rx=0 tx=0).stop 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 msgr2=0x7f0b240a27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c011940 tx=0x7f0b0c011e00 comp rx=0 tx=0).stop 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 shutdown_connections 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0b0406c8f0 0x7f0b0406edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b240c96d0 0x7f0b240c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0b2409fab0 0x7f0b240a27a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.234 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 --2- 172.21.15.62:0/76733787 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0b10004d40 0x7f0b240ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 >> 172.21.15.62:0/76733787 conn(0x7f0b2409e510 msgr2=0x7f0b240b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 shutdown_connections 2024-04-26T22:28:14.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:14.233+0000 7f0b29178640 1 -- 172.21.15.62:0/76733787 wait complete. 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T22:28:14.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T22:28:14.245 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:28:14.245 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:14.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.245 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:28:14.262 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f0 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f1 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42684sec preferred_lft 42684sec 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::daa0:5f76:a60a:7388/64 scope link noprefixroute 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.263 INFO:teuthology.orchestra.run.smithi174.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether da:52:8b:02:eb:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::d03c:ebff:fec0:f023/64 scope link 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether ea:9b:af:31:81:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::e89b:afff:fe31:8176/64 scope link 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether da:52:8b:02:eb:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::d852:8bff:fe02:ebe8/64 scope link 2024-04-26T22:28:14.264 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:14.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.265 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:28:14.325 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-04-26T22:28:14.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.327 DEBUG:teuthology.orchestra.run.smithi174:> route 2024-04-26T22:28:14.382 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2024-04-26T22:28:14.383 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:28:14.383 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:28:14.383 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:28:14.383 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:28:14.383 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:28:14.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:28:14.502 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:14.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.508 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:28:14.523 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T22:28:14.523 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T22:28:14.523 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:14.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.524 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.0 up 2024-04-26T22:28:14.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:14.589 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:14.645 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:14.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:14.645 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:14.700 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:28:14.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.700 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:14.781 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:14.782 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:14.782 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:14.790 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:14.790 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:14.790 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:14.790 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:14.790 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:14.795 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:14.795 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:14.795 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:14.795 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:28:14.796 INFO:teuthology.orchestra.run.smithi174.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:28:14.797 INFO:teuthology.orchestra.run.smithi174.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:28:14.797 INFO:teuthology.orchestra.run.smithi174.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:28:14.797 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:28:14.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:14.798 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:14.861 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:28:14.949 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:28:14.951 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:14.957 INFO:teuthology.orchestra.run.smithi174.stdout:client4841 2024-04-26T22:28:14.957 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:14.957 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:14.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T22:28:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.080+0000 7f5dae146640 1 Processor -- start 2024-04-26T22:28:15.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.080+0000 7f5dae146640 1 -- start start 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5dae146640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5da8146bb0 0x7f5da80c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5dae146640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da8071f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5dae146640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da80c6c40 con 0x7f5da80d01e0 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5dae146640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f5da80cf8d0 con 0x7f5da8146bb0 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5dae146640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80ce9e0 con 0x7f5da80cfd60 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5da8146bb0 0x7f5da80c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f5da8146bb0 0x7f5da80c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38462/0 (socket says 172.21.15.62:38462) 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 -- 172.21.15.62:0/2702139440 learned_addr learned my addr 172.21.15.62:0/2702139440 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da6ffd640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da8071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 -- 172.21.15.62:0/2702139440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 msgr2=0x7f5da8071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:15.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da8071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 -- 172.21.15.62:0/2702139440 >> v1:172.21.15.174:6789/0 conn(0x7f5da80d01e0 legacy=0x7f5da80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.083 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.081+0000 7f5da77fe640 1 -- 172.21.15.62:0/2702139440 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da80cecc0 con 0x7f5da8146bb0 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.082+0000 7f5da77fe640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80c6700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009870 tx=0x7f5d9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5d1a9f818aa68f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.082+0000 7f5da67fc640 1 -- 172.21.15.62:0/2702139440 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037070 con 0x7f5da8146bb0 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.082+0000 7f5da67fc640 1 -- 172.21.15.62:0/2702139440 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98032440 con 0x7f5da8146bb0 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.082+0000 7f5da67fc640 1 -- 172.21.15.62:0/2702139440 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98032810 con 0x7f5da8146bb0 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 -- 172.21.15.62:0/2702139440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 msgr2=0x7f5da80c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.084 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80c6700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009870 tx=0x7f5d9802a270 comp rx=0 tx=0).stop 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 -- 172.21.15.62:0/2702139440 shutdown_connections 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da8071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 --2- 172.21.15.62:0/2702139440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80c6700 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 -- 172.21.15.62:0/2702139440 >> 172.21.15.62:0/2702139440 conn(0x7f5da80b2c60 msgr2=0x7f5da80b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 -- 172.21.15.62:0/2702139440 shutdown_connections 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.083+0000 7f5dae146640 1 -- 172.21.15.62:0/2702139440 wait complete. 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.084+0000 7f5dae146640 1 Processor -- start 2024-04-26T22:28:15.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.084+0000 7f5dae146640 1 -- start start 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da80b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 0x7f5da8142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80cf560 con 0x7f5da8146bb0 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80cf280 con 0x7f5da80d01e0 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5dae146640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80cb5f0 con 0x7f5da80cfd60 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38478/0 (socket says 172.21.15.62:38478) 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da77fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da80b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 -- 172.21.15.62:0/1906685020 learned_addr learned my addr 172.21.15.62:0/1906685020 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 0x7f5da8142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 -- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 msgr2=0x7f5da80b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da80b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 -- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 msgr2=0x7f5da8142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 0x7f5da8142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da7fff640 1 -- 172.21.15.62:0/1906685020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8145900 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.085+0000 7f5da6ffd640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 0x7f5da8142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5da7fff640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5da000c730 tx=0x7f5da000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0003b70 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5da0003d30 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da8145be0 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0005310 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.086+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da80b5100 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.087+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8137fc0 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.087+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5da00056c0 con 0x7f5da8146bb0 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.088+0000 7f5d8ffff640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 0x7f5d9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.088+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f5da008ba00 con 0x7f5da8146bb0 2024-04-26T22:28:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.088+0000 7f5da77fe640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 0x7f5d9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.088+0000 7f5da77fe640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 0x7f5d9406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.092+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5da8137fc0 con 0x7f5da8146bb0 2024-04-26T22:28:15.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.317+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5da806a360 con 0x7f5da8146bb0 2024-04-26T22:28:15.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.722+0000 7f5d8ffff640 1 -- 172.21.15.62:0/1906685020 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7f5da0059870 con 0x7f5da8146bb0 2024-04-26T22:28:15.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 msgr2=0x7f5d9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 0x7f5d9406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).stop 2024-04-26T22:28:15.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 msgr2=0x7f5da80b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5da000c730 tx=0x7f5da000cbf0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 shutdown_connections 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5d9406c8f0 0x7f5d9406edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5da8146bb0 0x7f5da80b4bc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da80d01e0 0x7f5da8142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 --2- 172.21.15.62:0/1906685020 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5da80cfd60 0x7f5da80b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 >> 172.21.15.62:0/1906685020 conn(0x7f5da80b2c60 msgr2=0x7f5da80d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 shutdown_connections 2024-04-26T22:28:15.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.725+0000 7f5dae146640 1 -- 172.21.15.62:0/1906685020 wait complete. 2024-04-26T22:28:15.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T22:28:15.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.849+0000 7f668619e640 1 Processor -- start 2024-04-26T22:28:15.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 -- start start 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f668015f5d0 0x7f66800d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f66800d5b80 0x7f66800d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66800cfc10 con 0x7f66800d3670 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f66800cfe90 con 0x7f668015f5d0 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668619e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f66800d0170 con 0x7f66800d5b80 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668519c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f668015f5d0 0x7f66800d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.850+0000 7f668599d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f66800d3670 0x7f66800b1cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53488/0 (socket says 172.21.15.62:53488) 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668599d640 1 -- 172.21.15.62:0/2059743064 learned_addr learned my addr 172.21.15.62:0/2059743064 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668499b640 1 --2- 172.21.15.62:0/2059743064 >> v2:172.21.15.174:3300/0 conn(0x7f66800d5b80 0x7f66800d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668519c640 1 -- 172.21.15.62:0/2059743064 >> v2:172.21.15.174:3300/0 conn(0x7f66800d5b80 msgr2=0x7f66800d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668519c640 1 --2- 172.21.15.62:0/2059743064 >> v2:172.21.15.174:3300/0 conn(0x7f66800d5b80 0x7f66800d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668519c640 1 -- 172.21.15.62:0/2059743064 >> v1:172.21.15.62:6789/0 conn(0x7f66800d3670 legacy=0x7f66800b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.852 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f667ffff640 1 -- 172.21.15.62:0/2059743064 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708644163 0 0) 0x7f6670001490 con 0x7f66800d3670 2024-04-26T22:28:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.851+0000 7f668519c640 1 -- 172.21.15.62:0/2059743064 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66800d0450 con 0x7f668015f5d0 2024-04-26T22:28:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668519c640 1 --2- 172.21.15.62:0/2059743064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800d0c50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f66800b7b90 tx=0x7f667802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5084b1af2d73d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f667ffff640 1 -- 172.21.15.62:0/2059743064 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678037070 con 0x7f668015f5d0 2024-04-26T22:28:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f667ffff640 1 -- 172.21.15.62:0/2059743064 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6678032440 con 0x7f668015f5d0 2024-04-26T22:28:15.853 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f667ffff640 1 -- 172.21.15.62:0/2059743064 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667802ae60 con 0x7f668015f5d0 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 -- 172.21.15.62:0/2059743064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 msgr2=0x7f66800d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 --2- 172.21.15.62:0/2059743064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800d0c50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f66800b7b90 tx=0x7f667802a110 comp rx=0 tx=0).stop 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 -- 172.21.15.62:0/2059743064 shutdown_connections 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 --2- 172.21.15.62:0/2059743064 >> v2:172.21.15.174:3300/0 conn(0x7f66800d5b80 0x7f66800d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 --2- 172.21.15.62:0/2059743064 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800d0c50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 -- 172.21.15.62:0/2059743064 >> 172.21.15.62:0/2059743064 conn(0x7f66800d5090 msgr2=0x7f66800d5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 -- 172.21.15.62:0/2059743064 shutdown_connections 2024-04-26T22:28:15.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.852+0000 7f668619e640 1 -- 172.21.15.62:0/2059743064 wait complete. 2024-04-26T22:28:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 Processor -- start 2024-04-26T22:28:15.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 -- start start 2024-04-26T22:28:15.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66800d3670 0x7f66800ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66800d5b80 0x7f66801566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.857 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66800b4350 con 0x7f668015f5d0 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66800b41d0 con 0x7f66800d5b80 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.854+0000 7f668619e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f66800d3ac0 con 0x7f66800d3670 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38496/0 (socket says 172.21.15.62:38496) 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 -- 172.21.15.62:0/2627376863 learned_addr learned my addr 172.21.15.62:0/2627376863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:15.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668499b640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66800d5b80 0x7f66801566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668519c640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66800d3670 0x7f66800ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 -- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66800d3670 msgr2=0x7f66800ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66800d3670 0x7f66800ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 -- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66800d5b80 msgr2=0x7f66801566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66800d5b80 0x7f66801566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 -- 172.21.15.62:0/2627376863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66800b8180 con 0x7f668015f5d0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668599d640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f667000c820 tx=0x7f667000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670018070 con 0x7f668015f5d0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f667000a5e0 con 0x7f668015f5d0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667000a9d0 con 0x7f668015f5d0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66800b8450 con 0x7f668015f5d0 2024-04-26T22:28:15.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.855+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66800b8a10 con 0x7f668015f5d0 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.856+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680076890 con 0x7f668015f5d0 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.857+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f667000ab70 con 0x7f668015f5d0 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.857+0000 7f667dffb640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.857+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f667008b060 con 0x7f668015f5d0 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.857+0000 7f668519c640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:15.860 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.858+0000 7f668519c640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 0x7f666c06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6678034000 tx=0x7f667802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:15.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:15.861+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66800cfc10 con 0x7f668015f5d0 2024-04-26T22:28:16.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.071+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f66800d3450 con 0x7f666c06c8f0 2024-04-26T22:28:16.073 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.073+0000 7f667dffb640 1 -- 172.21.15.62:0/2627376863 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f66800d3450 con 0x7f666c06c8f0 2024-04-26T22:28:16.073 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-26T22:28:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephfs" 2024-04-26T22:28:16.073 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:16.074 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-26T22:28:16.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 msgr2=0x7f666c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 0x7f666c06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6678034000 tx=0x7f667802e000 comp rx=0 tx=0).stop 2024-04-26T22:28:16.075 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 msgr2=0x7f66800b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f667000c820 tx=0x7f667000cce0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 shutdown_connections 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f666c06c8f0 0x7f666c06edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f668015f5d0 0x7f66800b7c40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66800d5b80 0x7f66801566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 --2- 172.21.15.62:0/2627376863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66800d3670 0x7f66800ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 >> 172.21.15.62:0/2627376863 conn(0x7f66800d5090 msgr2=0x7f66800d1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 shutdown_connections 2024-04-26T22:28:16.076 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.075+0000 7f668619e640 1 -- 172.21.15.62:0/2627376863 wait complete. 2024-04-26T22:28:16.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T22:28:16.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.190+0000 7fed1282a640 1 Processor -- start 2024-04-26T22:28:16.191 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.190+0000 7fed1282a640 1 -- start start 2024-04-26T22:28:16.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed1282a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fed0c0cfba0 0x7fed0c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed1282a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed0c15d830 0x7fed0c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed1282a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0c0cdb90 con 0x7fed0c0b25e0 2024-04-26T22:28:16.192 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed1282a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0b2390 con 0x7fed0c15d830 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed1282a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0d1ff0 con 0x7fed0c0cfba0 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed10dc7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed0c15d830 0x7fed0c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed10dc7640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fed0c15d830 0x7fed0c0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38506/0 (socket says 172.21.15.62:38506) 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed10dc7640 1 -- 172.21.15.62:0/2324304081 learned_addr learned my addr 172.21.15.62:0/2324304081 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.191+0000 7fed0bfff640 1 --2- 172.21.15.62:0/2324304081 >> v2:172.21.15.174:3300/0 conn(0x7fed0c0cfba0 0x7fed0c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed10dc7640 1 -- 172.21.15.62:0/2324304081 >> v2:172.21.15.174:3300/0 conn(0x7fed0c0cfba0 msgr2=0x7fed0c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed10dc7640 1 --2- 172.21.15.62:0/2324304081 >> v2:172.21.15.174:3300/0 conn(0x7fed0c0cfba0 0x7fed0c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed10dc7640 1 -- 172.21.15.62:0/2324304081 >> v1:172.21.15.174:6789/0 conn(0x7fed0c0b25e0 legacy=0x7fed0c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed10dc7640 1 -- 172.21.15.62:0/2324304081 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0d1a30 con 0x7fed0c15d830 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed10dc7640 1 --2- 172.21.15.62:0/2324304081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0cff80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0d2ee0 tx=0x7fed0402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17b96d42beb6c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed0affd640 1 -- 172.21.15.62:0/2324304081 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04036070 con 0x7fed0c15d830 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed0affd640 1 -- 172.21.15.62:0/2324304081 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed04031600 con 0x7fed0c15d830 2024-04-26T22:28:16.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.192+0000 7fed0affd640 1 -- 172.21.15.62:0/2324304081 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed040319d0 con 0x7fed0c15d830 2024-04-26T22:28:16.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 -- 172.21.15.62:0/2324304081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 msgr2=0x7fed0c0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 --2- 172.21.15.62:0/2324304081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0cff80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0d2ee0 tx=0x7fed0402a660 comp rx=0 tx=0).stop 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 -- 172.21.15.62:0/2324304081 shutdown_connections 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 --2- 172.21.15.62:0/2324304081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0cff80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 --2- 172.21.15.62:0/2324304081 >> v2:172.21.15.174:3300/0 conn(0x7fed0c0cfba0 0x7fed0c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 -- 172.21.15.62:0/2324304081 >> 172.21.15.62:0/2324304081 conn(0x7fed0c0d2f80 msgr2=0x7fed0c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 -- 172.21.15.62:0/2324304081 shutdown_connections 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.193+0000 7fed1282a640 1 -- 172.21.15.62:0/2324304081 wait complete. 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.194+0000 7fed1282a640 1 Processor -- start 2024-04-26T22:28:16.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.194+0000 7fed1282a640 1 -- start start 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed0c0b25e0 0x7fed0c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 0x7fed0c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c0d3aa0 con 0x7fed0c15d830 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c0cde30 con 0x7fed0c0cfba0 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed1282a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c0d3c20 con 0x7fed0c0b25e0 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.197 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38520/0 (socket says 172.21.15.62:38520) 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 -- 172.21.15.62:0/960316871 learned_addr learned my addr 172.21.15.62:0/960316871 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed0b7fe640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 0x7fed0c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed0bfff640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed0c0b25e0 0x7fed0c0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 -- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed0c0b25e0 msgr2=0x7fed0c0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed0c0b25e0 0x7fed0c0cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 -- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 msgr2=0x7fed0c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 0x7fed0c1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed10dc7640 1 -- 172.21.15.62:0/960316871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0c9220 con 0x7fed0c15d830 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.195+0000 7fed0b7fe640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 0x7fed0c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed10dc7640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed04009f40 tx=0x7fed04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0403f020 con 0x7fed0c15d830 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed04037040 con 0x7fed0c15d830 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c0c9500 con 0x7fed0c15d830 2024-04-26T22:28:16.198 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04036070 con 0x7fed0c15d830 2024-04-26T22:28:16.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.196+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c15f050 con 0x7fed0c15d830 2024-04-26T22:28:16.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.197+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed0c15f250 con 0x7fed0c15d830 2024-04-26T22:28:16.199 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.198+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed04042020 con 0x7fed0c15d830 2024-04-26T22:28:16.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.198+0000 7fed08ff9640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 0x7fecf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.198+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fed040ac2a0 con 0x7fed0c15d830 2024-04-26T22:28:16.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.198+0000 7fed0bfff640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 0x7fecf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.200 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.199+0000 7fed0bfff640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 0x7fecf806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecfc006fd0 tx=0x7fecfc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.202 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.202+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed0c15f250 con 0x7fed0c15d830 2024-04-26T22:28:16.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.385+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fed0c0cf470 con 0x7fed0c15d830 2024-04-26T22:28:16.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.386+0000 7fed08ff9640 1 -- 172.21.15.62:0/960316871 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fed04049020 con 0x7fed0c15d830 2024-04-26T22:28:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 msgr2=0x7fecf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 0x7fecf806edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecfc006fd0 tx=0x7fecfc008040 comp rx=0 tx=0).stop 2024-04-26T22:28:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 msgr2=0x7fed0c0c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed04009f40 tx=0x7fed04004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:16.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 shutdown_connections 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fecf806c8f0 0x7fecf806edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fed0c15d830 0x7fed0c0c8960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c0cfba0 0x7fed0c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 --2- 172.21.15.62:0/960316871 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fed0c0b25e0 0x7fed0c0cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 >> 172.21.15.62:0/960316871 conn(0x7fed0c0d2f80 msgr2=0x7fed0c070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 shutdown_connections 2024-04-26T22:28:16.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.388+0000 7fed1282a640 1 -- 172.21.15.62:0/960316871 wait complete. 2024-04-26T22:28:16.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp94470 2024-04-26T22:28:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.503+0000 7fe20c072640 1 Processor -- start 2024-04-26T22:28:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.503+0000 7fe20c072640 1 -- start start 2024-04-26T22:28:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20c072640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe2041610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.510 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20c072640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2040d5f80 0x7fe2040d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20c072640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2040d0930 con 0x7fe2040d5b60 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20c072640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe2040d0c00 con 0x7fe2040d5f80 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20c072640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe2040d0ee0 con 0x7fe204160800 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20a60f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2040d5f80 0x7fe2040d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20a60f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2040d5f80 0x7fe2040d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38524/0 (socket says 172.21.15.62:38524) 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.504+0000 7fe20a60f640 1 -- 172.21.15.62:0/2564082007 learned_addr learned my addr 172.21.15.62:0/2564082007 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.505+0000 7fe209e0e640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe2041610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.505+0000 7fe20a60f640 1 -- 172.21.15.62:0/2564082007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 msgr2=0x7fe2041610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.505+0000 7fe20a60f640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe2041610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.505+0000 7fe20a60f640 1 -- 172.21.15.62:0/2564082007 >> v1:172.21.15.174:6789/0 conn(0x7fe2040d5b60 legacy=0x7fe2040d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.505+0000 7fe20a60f640 1 -- 172.21.15.62:0/2564082007 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2040d11c0 con 0x7fe2040d5f80 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20a60f640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d5f80 0x7fe2040d3610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82964eb229ba26f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe208e0c640 1 -- 172.21.15.62:0/2564082007 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4036070 con 0x7fe2040d5f80 2024-04-26T22:28:16.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe208e0c640 1 -- 172.21.15.62:0/2564082007 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f4031600 con 0x7fe2040d5f80 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe208e0c640 1 -- 172.21.15.62:0/2564082007 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40319d0 con 0x7fe2040d5f80 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 -- 172.21.15.62:0/2564082007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d5f80 msgr2=0x7fe2040d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d5f80 0x7fe2040d3610 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f402a710 comp rx=0 tx=0).stop 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 -- 172.21.15.62:0/2564082007 shutdown_connections 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d5f80 0x7fe2040d3610 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 --2- 172.21.15.62:0/2564082007 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe2041610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 -- 172.21.15.62:0/2564082007 >> 172.21.15.62:0/2564082007 conn(0x7fe20409e550 msgr2=0x7fe2040d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 -- 172.21.15.62:0/2564082007 shutdown_connections 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.506+0000 7fe20c072640 1 -- 172.21.15.62:0/2564082007 wait complete. 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.507+0000 7fe20c072640 1 Processor -- start 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 -- start start 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040d5b60 0x7fe204158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe20415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040b6300 con 0x7fe2040d1ab0 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040d30f0 con 0x7fe2040d5b60 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20c072640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe204160be0 con 0x7fe204160800 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20a60f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20a60f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38528/0 (socket says 172.21.15.62:38528) 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.508+0000 7fe20a60f640 1 -- 172.21.15.62:0/954113143 learned_addr learned my addr 172.21.15.62:0/954113143 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe209e0e640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040d5b60 0x7fe204158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20960d640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe20415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 -- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 msgr2=0x7fe20415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe20415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 -- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040d5b60 msgr2=0x7fe204158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040d5b60 0x7fe204158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 -- 172.21.15.62:0/954113143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe204159030 con 0x7fe2040d1ab0 2024-04-26T22:28:16.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20a60f640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009f70 tx=0x7fe1f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f403f020 con 0x7fe2040d1ab0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f4037040 con 0x7fe2040d1ab0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2041f6790 con 0x7fe2040d1ab0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4036070 con 0x7fe2040d1ab0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.509+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20415b830 con 0x7fe2040d1ab0 2024-04-26T22:28:16.514 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.510+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20406a360 con 0x7fe2040d1ab0 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.510+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1f4042020 con 0x7fe2040d1ab0 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.511+0000 7fe2027fc640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.511+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fe1f40ac020 con 0x7fe2040d1ab0 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.511+0000 7fe209e0e640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.512+0000 7fe209e0e640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 0x7fe1f006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.515 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.515+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1f4049030 con 0x7fe2040d1ab0 2024-04-26T22:28:16.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.726+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp94470", "target": ["mon-mgr", ""]}) -- 0x7fe2040d4530 con 0x7fe1f006c8f0 2024-04-26T22:28:16.732 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:16.731+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:16.732 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:16.731+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:16.732 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:16.731+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:16.732 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:16.731+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:16.732 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:16.731+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:16.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.750+0000 7fe2027fc640 1 -- 172.21.15.62:0/954113143 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe2040d4530 con 0x7fe1f006c8f0 2024-04-26T22:28:16.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 msgr2=0x7fe1f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 0x7fe1f006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc006fd0 tx=0x7fe1fc008040 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 msgr2=0x7fe2040d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009f70 tx=0x7fe1f4004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 shutdown_connections 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c8f0 0x7fe1f006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d1ab0 0x7fe2040d1f40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204160800 0x7fe20415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 --2- 172.21.15.62:0/954113143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040d5b60 0x7fe204158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 >> 172.21.15.62:0/954113143 conn(0x7fe20409e550 msgr2=0x7fe2041610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 shutdown_connections 2024-04-26T22:28:16.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.752+0000 7fe20c072640 1 -- 172.21.15.62:0/954113143 wait complete. 2024-04-26T22:28:16.763 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp94470 subvol_snap94290 2024-04-26T22:28:16.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.869+0000 7efdc6122640 1 Processor -- start 2024-04-26T22:28:16.870 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.869+0000 7efdc6122640 1 -- start start 2024-04-26T22:28:16.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdc6122640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00d4f30 0x7efdc00d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdc6122640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efdc00b58b0 0x7efdc00b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.871 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdc6122640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc00b50f0 con 0x7efdc015f5d0 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdc6122640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7efdc00b5420 con 0x7efdc00b58b0 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdc6122640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00d58d0 con 0x7efdc00d4f30 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efdc00b58b0 0x7efdc00b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7efdc00b58b0 0x7efdc00b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38538/0 (socket says 172.21.15.62:38538) 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 -- 172.21.15.62:0/1700369831 learned_addr learned my addr 172.21.15.62:0/1700369831 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbeffd640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00d4f30 0x7efdc00d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 -- 172.21.15.62:0/1700369831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00d4f30 msgr2=0x7efdc00d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00d4f30 0x7efdc00d5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.870+0000 7efdbffff640 1 -- 172.21.15.62:0/1700369831 >> v1:172.21.15.174:6789/0 conn(0x7efdc015f5d0 legacy=0x7efdc00d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:16.872 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.871+0000 7efdbffff640 1 -- 172.21.15.62:0/1700369831 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc00d0a80 con 0x7efdc00b58b0 2024-04-26T22:28:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.871+0000 7efdbffff640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc00b58b0 0x7efdc00b5d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e50 tx=0x7efdb802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b2eda4d5578c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.871+0000 7efdbe7fc640 1 -- 172.21.15.62:0/1700369831 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8036070 con 0x7efdc00b58b0 2024-04-26T22:28:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.871+0000 7efdbe7fc640 1 -- 172.21.15.62:0/1700369831 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb8031600 con 0x7efdc00b58b0 2024-04-26T22:28:16.873 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.871+0000 7efdbe7fc640 1 -- 172.21.15.62:0/1700369831 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb80319d0 con 0x7efdc00b58b0 2024-04-26T22:28:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 -- 172.21.15.62:0/1700369831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc00b58b0 msgr2=0x7efdc00b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.875 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc00b58b0 0x7efdc00b5d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e50 tx=0x7efdb802a7a0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 -- 172.21.15.62:0/1700369831 shutdown_connections 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc00b58b0 0x7efdc00b5d40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 --2- 172.21.15.62:0/1700369831 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00d4f30 0x7efdc00d5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 -- 172.21.15.62:0/1700369831 >> 172.21.15.62:0/1700369831 conn(0x7efdc008ebd0 msgr2=0x7efdc009e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 -- 172.21.15.62:0/1700369831 shutdown_connections 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.872+0000 7efdc6122640 1 -- 172.21.15.62:0/1700369831 wait complete. 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.873+0000 7efdc6122640 1 Processor -- start 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.873+0000 7efdc6122640 1 -- start start 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00b58b0 0x7efdc00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdc00d4f30 0x7efdc00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc015f5d0 0x7efdc01e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efdc015f9b0 con 0x7efdc015f5d0 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdc00d1040 con 0x7efdc00d4f30 2024-04-26T22:28:16.876 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdc6122640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efdc00d41e0 con 0x7efdc00b58b0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc015f5d0 0x7efdc01e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc015f5d0 0x7efdc01e7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38542/0 (socket says 172.21.15.62:38542) 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbffff640 1 -- 172.21.15.62:0/3156451496 learned_addr learned my addr 172.21.15.62:0/3156451496 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbeffd640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdc00d4f30 0x7efdc00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbf7fe640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00b58b0 0x7efdc00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbffff640 1 -- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00b58b0 msgr2=0x7efdc00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.874+0000 7efdbffff640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdc00b58b0 0x7efdc00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdbffff640 1 -- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdc00d4f30 msgr2=0x7efdc00cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdbffff640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdc00d4f30 0x7efdc00cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdbffff640 1 -- 172.21.15.62:0/3156451496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc014c7b0 con 0x7efdc015f5d0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdbffff640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efdc015f5d0 0x7efdc01e7c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e50 tx=0x7efdb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8041020 con 0x7efdc015f5d0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb8007780 con 0x7efdc015f5d0 2024-04-26T22:28:16.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdc6122640 1 -- 172.21.15.62:0/3156451496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc014ca50 con 0x7efdc015f5d0 2024-04-26T22:28:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8036040 con 0x7efdc015f5d0 2024-04-26T22:28:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.875+0000 7efdc6122640 1 -- 172.21.15.62:0/3156451496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc014d010 con 0x7efdc015f5d0 2024-04-26T22:28:16.878 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.876+0000 7efdc6122640 1 -- 172.21.15.62:0/3156451496 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc006a360 con 0x7efdc015f5d0 2024-04-26T22:28:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.876+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb8044020 con 0x7efdc015f5d0 2024-04-26T22:28:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.877+0000 7efda7fff640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7efdac06c8f0 0x7efdac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.877+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7efdb80ac6b0 con 0x7efdc015f5d0 2024-04-26T22:28:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.877+0000 7efdbf7fe640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7efdac06c8f0 0x7efdac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:16.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.877+0000 7efdbf7fe640 1 --2- 172.21.15.62:0/3156451496 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7efdac06c8f0 0x7efdac06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdb00016e0 tx=0x7efdb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:16.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:16.881+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdb804b030 con 0x7efdc015f5d0 2024-04-26T22:28:17.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.093+0000 7efdc6122640 1 -- 172.21.15.62:0/3156451496 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp94470", "snap_name": "subvol_snap94290", "target": ["mon-mgr", ""]}) -- 0x7efdc00b61c0 con 0x7efdac06c8f0 2024-04-26T22:28:17.094 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:17.094+0000 7f9cbb4ea640 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2024-04-26T22:28:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.095+0000 7efda7fff640 1 -- 172.21.15.62:0/3156451496 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) ==== 51+0+0 (secure 0 0 0) 0x7efdc00b61c0 con 0x7efdac06c8f0 2024-04-26T22:28:17.095 INFO:teuthology.orchestra.run.smithi062.stderr:Error ENOSYS: subvolume group snapshots are not supported 2024-04-26T22:28:17.100 DEBUG:teuthology.orchestra.run:got remote process result: 38 2024-04-26T22:28:17.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp94470 2024-04-26T22:28:17.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.204+0000 7ff1f0bc6640 1 Processor -- start 2024-04-26T22:28:17.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 -- start start 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1ec0cff80 0x7ff1ec0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1ec0cd640 con 0x7ff1ec0b2290 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff1ec0cd920 con 0x7ff1ec0cff80 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.205+0000 7ff1f0bc6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec0cdc00 con 0x7ff1ec0d1fe0 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1ec0cff80 0x7ff1ec0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff1ec0cff80 0x7ff1ec0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38548/0 (socket says 172.21.15.62:38548) 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2729302206 learned_addr learned my addr 172.21.15.62:0/2729302206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1ead9d640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2729302206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0d1fe0 msgr2=0x7ff1ec0d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2729302206 >> v1:172.21.15.174:6789/0 conn(0x7ff1ec0b2290 legacy=0x7ff1ec0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.206+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2729302206 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec0cdee0 con 0x7ff1ec0cff80 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1e9d9b640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b2930 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800b6e045c9c280a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1e959a640 1 -- 172.21.15.62:0/2729302206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037070 con 0x7ff1ec0cff80 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1e959a640 1 -- 172.21.15.62:0/2729302206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0032440 con 0x7ff1ec0cff80 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1e959a640 1 -- 172.21.15.62:0/2729302206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0032810 con 0x7ff1ec0cff80 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2729302206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 msgr2=0x7ff1ec0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b2930 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002a270 comp rx=0 tx=0).stop 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2729302206 shutdown_connections 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2729302206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b2930 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2729302206 >> 172.21.15.62:0/2729302206 conn(0x7ff1ec0d2ff0 msgr2=0x7ff1ec0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2729302206 shutdown_connections 2024-04-26T22:28:17.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.207+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2729302206 wait complete. 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.209+0000 7ff1f0bc6640 1 Processor -- start 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.209+0000 7ff1f0bc6640 1 -- start start 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.209+0000 7ff1f0bc6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 0x7ff1ec0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.209+0000 7ff1f0bc6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0d2950 con 0x7ff1ec0cff80 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec0b2670 con 0x7ff1ec0d1fe0 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec0d2ad0 con 0x7ff1ec0b2290 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38560/0 (socket says 172.21.15.62:38560) 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2622151209 learned_addr learned my addr 172.21.15.62:0/2622151209 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1ea59c640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 0x7ff1ec0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1ead9d640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 msgr2=0x7ff1ec0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 0x7ff1ec0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1ec0d1fe0 msgr2=0x7ff1ec0b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 -- 172.21.15.62:0/2622151209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec160e00 con 0x7ff1ec0cff80 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1ea59c640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 0x7ff1ec0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1e9d9b640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a2b0 tx=0x7ff1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037040 con 0x7ff1ec0cff80 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0042070 con 0x7ff1ec0cff80 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1610e0 con 0x7ff1ec0cff80 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0030880 con 0x7ff1ec0cff80 2024-04-26T22:28:17.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.210+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1616a0 con 0x7ff1ec0cff80 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.211+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec06a360 con 0x7ff1ec0cff80 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.212+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1e0030a20 con 0x7ff1ec0cff80 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.212+0000 7ff1daffd640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.212+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ff1e00ac2e0 con 0x7ff1ec0cff80 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.212+0000 7ff1ea59c640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.213+0000 7ff1ea59c640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc001700 tx=0x7ff1dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.216+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1e007a7a0 con 0x7ff1ec0cff80 2024-04-26T22:28:17.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.426+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp94470", "target": ["mon-mgr", ""]}) -- 0x7ff1ec0ce880 con 0x7ff1cc06c8f0 2024-04-26T22:28:17.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.430+0000 7ff1daffd640 1 -- 172.21.15.62:0/2622151209 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff1ec0ce880 con 0x7ff1cc06c8f0 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 msgr2=0x7ff1cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc001700 tx=0x7ff1dc009040 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 msgr2=0x7ff1ec0b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a2b0 tx=0x7ff1e00046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 shutdown_connections 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1cc06c8f0 0x7ff1cc06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1ec0d1fe0 0x7ff1ec0b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1ec0cff80 0x7ff1ec0b8110 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 --2- 172.21.15.62:0/2622151209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1ec0b2290 0x7ff1ec0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 >> 172.21.15.62:0/2622151209 conn(0x7ff1ec0d2ff0 msgr2=0x7ff1ec0d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:17.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 shutdown_connections 2024-04-26T22:28:17.434 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.432+0000 7ff1f0bc6640 1 -- 172.21.15.62:0/2622151209 wait complete. 2024-04-26T22:28:17.443 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T22:28:17.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:28:17.481 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:28:17.531 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T22:28:17.532 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T22:28:17.532 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:17.532 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:17.548 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:17.548 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:17.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:17.746 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.746+0000 7fcea282c640 1 Processor -- start 2024-04-26T22:28:17.747 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.746+0000 7fcea282c640 1 -- start start 2024-04-26T22:28:17.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c162e40 0x7fce9c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fce9c0b2050 0x7fce9c0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9c0cfd70 0x7fce9c0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.748 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c163220 con 0x7fce9c0cfd70 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fce9c0d0eb0 con 0x7fce9c0b2050 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea282c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0d01a0 con 0x7fce9c162e40 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9c0cfd70 0x7fce9c0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fce9c0cfd70 0x7fce9c0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38574/0 (socket says 172.21.15.62:38574) 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 -- 172.21.15.62:0/132836 learned_addr learned my addr 172.21.15.62:0/132836 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fce9bfff640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c162e40 0x7fce9c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fce9b7fe640 1 --2- 172.21.15.62:0/132836 >> v2:172.21.15.174:3300/0 conn(0x7fce9c0b2050 0x7fce9c0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 -- 172.21.15.62:0/132836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c162e40 msgr2=0x7fce9c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c162e40 0x7fce9c0d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 -- 172.21.15.62:0/132836 >> v2:172.21.15.174:3300/0 conn(0x7fce9c0b2050 msgr2=0x7fce9c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.747+0000 7fcea0dc9640 1 --2- 172.21.15.62:0/132836 >> v2:172.21.15.174:3300/0 conn(0x7fce9c0b2050 0x7fce9c0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.748+0000 7fcea0dc9640 1 -- 172.21.15.62:0/132836 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0b1d70 con 0x7fce9c0cfd70 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.748+0000 7fcea0dc9640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c0ce010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce9c073b00 tx=0x7fce9402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed035c373b2f645 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.749 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.748+0000 7fce9affd640 1 -- 172.21.15.62:0/132836 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94036070 con 0x7fce9c0cfd70 2024-04-26T22:28:17.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.748+0000 7fce9affd640 1 -- 172.21.15.62:0/132836 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce94031600 con 0x7fce9c0cfd70 2024-04-26T22:28:17.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.748+0000 7fce9affd640 1 -- 172.21.15.62:0/132836 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce940319d0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 -- 172.21.15.62:0/132836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 msgr2=0x7fce9c0ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c0ce010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fce9c073b00 tx=0x7fce9402a610 comp rx=0 tx=0).stop 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 -- 172.21.15.62:0/132836 shutdown_connections 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c0ce010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 --2- 172.21.15.62:0/132836 >> v2:172.21.15.174:3300/0 conn(0x7fce9c0b2050 0x7fce9c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 --2- 172.21.15.62:0/132836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c162e40 0x7fce9c0d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 -- 172.21.15.62:0/132836 >> 172.21.15.62:0/132836 conn(0x7fce9c09ba50 msgr2=0x7fce9c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 -- 172.21.15.62:0/132836 shutdown_connections 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.749+0000 7fcea282c640 1 -- 172.21.15.62:0/132836 wait complete. 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.750+0000 7fcea282c640 1 Processor -- start 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.750+0000 7fcea282c640 1 -- start start 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c0b2050 0x7fce9c15c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce9c162e40 0x7fce9c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0d1f20 con 0x7fce9c0cfd70 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c0d1c40 con 0x7fce9c162e40 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea282c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce9c0b24b0 con 0x7fce9c0b2050 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38588/0 (socket says 172.21.15.62:38588) 2024-04-26T22:28:17.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 -- 172.21.15.62:0/3888131877 learned_addr learned my addr 172.21.15.62:0/3888131877 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9bfff640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c0b2050 0x7fce9c15c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fcea0dc9640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce9c162e40 0x7fce9c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 -- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c0b2050 msgr2=0x7fce9c15c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c0b2050 0x7fce9c15c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 -- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce9c162e40 msgr2=0x7fce9c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce9c162e40 0x7fce9c0ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.751+0000 7fce9b7fe640 1 -- 172.21.15.62:0/3888131877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c0c8800 con 0x7fce9c0cfd70 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fce9b7fe640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d800 tx=0x7fce8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c018030 con 0x7fce9c0cfd70 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c0096a0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c009870 con 0x7fce9c0cfd70 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c0c8ae0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.752+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c0c90a0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.753+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c06a360 con 0x7fce9c0cfd70 2024-04-26T22:28:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.753+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fce8c01b070 con 0x7fce9c0cfd70 2024-04-26T22:28:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.754+0000 7fce98ff9640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 0x7fce7c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:17.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.754+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fce8c08bce0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.754+0000 7fce9bfff640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 0x7fce7c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:17.756 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.754+0000 7fce9bfff640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 0x7fce7c06ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:17.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.758+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce8c059ad0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.944+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce9c06d580 con 0x7fce9c0cfd70 2024-04-26T22:28:17.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.945+0000 7fce98ff9640 1 -- 172.21.15.62:0/3888131877 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7fce8c05ddd0 con 0x7fce9c0cfd70 2024-04-26T22:28:17.946 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:17.946 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:17.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 msgr2=0x7fce7c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 0x7fce7c06ee40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce84006fd0 tx=0x7fce84008040 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 msgr2=0x7fce9c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00d800 tx=0x7fce8c00dcc0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 shutdown_connections 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce7c06c980 0x7fce7c06ee40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce9c162e40 0x7fce9c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce9c0cfd70 0x7fce9c160380 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 --2- 172.21.15.62:0/3888131877 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce9c0b2050 0x7fce9c15c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 >> 172.21.15.62:0/3888131877 conn(0x7fce9c09ba50 msgr2=0x7fce9c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 shutdown_connections 2024-04-26T22:28:17.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:17.947+0000 7fcea282c640 1 -- 172.21.15.62:0/3888131877 wait complete. 2024-04-26T22:28:17.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:28:00.329554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:28:00.858734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:28:17.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:28:18.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.062+0000 7f3103a64640 1 Processor -- start 2024-04-26T22:28:18.062 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.062+0000 7f3103a64640 1 -- start start 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3103a64640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f30fc0b2290 0x7f30fc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3103a64640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc0cd640 con 0x7f30fc0cf870 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3103a64640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30fc0cd920 con 0x7f30fc0d26f0 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3103a64640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f30fc0d2b20 con 0x7f30fc0b2290 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3100fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f30fc0cf870 0x7f30fc0d1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53504/0 (socket says 172.21.15.62:53504) 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3100fff640 1 -- 172.21.15.62:0/3355013334 learned_addr learned my addr 172.21.15.62:0/3355013334 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.063+0000 7f3101800640 1 --2- 172.21.15.62:0/3355013334 >> v2:172.21.15.174:3300/0 conn(0x7f30fc0b2290 0x7f30fc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.064+0000 7f30f3fff640 1 -- 172.21.15.62:0/3355013334 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852381990 0 0) 0x7f30fc0cd640 con 0x7f30fc0cf870 2024-04-26T22:28:18.064 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.064+0000 7f30f3fff640 1 -- 172.21.15.62:0/3355013334 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30e8003520 con 0x7f30fc0cf870 2024-04-26T22:28:18.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.064+0000 7f3101800640 1 -- 172.21.15.62:0/3355013334 >> v1:172.21.15.174:6789/0 conn(0x7f30fc0d26f0 legacy=0x7f30fc15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.064+0000 7f3101800640 1 -- 172.21.15.62:0/3355013334 >> v1:172.21.15.62:6789/0 conn(0x7f30fc0cf870 legacy=0x7f30fc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.065 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.064+0000 7f3101800640 1 -- 172.21.15.62:0/3355013334 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc0cdc00 con 0x7f30fc0b2290 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3101800640 1 --2- 172.21.15.62:0/3355013334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0b2290 0x7f30fc0b2930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009d90 tx=0x7f30f802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb36b6e7bd7eeb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f30f3fff640 1 -- 172.21.15.62:0/3355013334 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8007740 con 0x7f30fc0b2290 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f30f3fff640 1 -- 172.21.15.62:0/3355013334 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f80078e0 con 0x7f30fc0b2290 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f30f3fff640 1 -- 172.21.15.62:0/3355013334 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8007cb0 con 0x7f30fc0b2290 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3103a64640 1 -- 172.21.15.62:0/3355013334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0b2290 msgr2=0x7f30fc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3103a64640 1 --2- 172.21.15.62:0/3355013334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0b2290 0x7f30fc0b2930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009d90 tx=0x7f30f802a6f0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3103a64640 1 -- 172.21.15.62:0/3355013334 shutdown_connections 2024-04-26T22:28:18.066 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3103a64640 1 --2- 172.21.15.62:0/3355013334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0b2290 0x7f30fc0b2930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.065+0000 7f3103a64640 1 -- 172.21.15.62:0/3355013334 >> 172.21.15.62:0/3355013334 conn(0x7f30fc0d2ff0 msgr2=0x7f30fc0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.066+0000 7f3103a64640 1 -- 172.21.15.62:0/3355013334 shutdown_connections 2024-04-26T22:28:18.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.066+0000 7f3103a64640 1 -- 172.21.15.62:0/3355013334 wait complete. 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 Processor -- start 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 -- start start 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 0x7f30fc0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 0x7f30fc0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc0b2670 con 0x7f30fc0b2290 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc0ce1c0 con 0x7f30fc0cf870 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3103a64640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc0cfd00 con 0x7f30fc0d26f0 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.067+0000 7f3101800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38602/0 (socket says 172.21.15.62:38602) 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 -- 172.21.15.62:0/4019279356 learned_addr learned my addr 172.21.15.62:0/4019279356 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 -- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 msgr2=0x7f30fc0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3100fff640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 0x7f30fc0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.069 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3102001640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 0x7f30fc0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 0x7f30fc0b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 -- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 msgr2=0x7f30fc0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 0x7f30fc0ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 -- 172.21.15.62:0/4019279356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc15bd50 con 0x7f30fc0b2290 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3102001640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 0x7f30fc0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3100fff640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 0x7f30fc0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3101800640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009d90 tx=0x7f30f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8040070 con 0x7f30fc0b2290 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30f8031c40 con 0x7f30fc0b2290 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc0ca310 con 0x7f30fc0b2290 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8009450 con 0x7f30fc0b2290 2024-04-26T22:28:18.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.068+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc0b6d80 con 0x7f30fc0b2290 2024-04-26T22:28:18.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.069+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc06a360 con 0x7f30fc0b2290 2024-04-26T22:28:18.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.070+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f30f8039040 con 0x7f30fc0b2290 2024-04-26T22:28:18.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.070+0000 7f30f1ffb640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 0x7f30e006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.070+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f30f80ad1c0 con 0x7f30fc0b2290 2024-04-26T22:28:18.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.070+0000 7f3100fff640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 0x7f30e006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.071+0000 7f3100fff640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 0x7f30e006ee40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0cc1e0 tx=0x7f30ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.074 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.074+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f30fc15bd50 con 0x7f30fc0b2290 2024-04-26T22:28:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.258+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30fc0b6b30 con 0x7f30fc0b2290 2024-04-26T22:28:18.258 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.258+0000 7f30f1ffb640 1 -- 172.21.15.62:0/4019279356 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f30f807aea0 con 0x7f30fc0b2290 2024-04-26T22:28:18.258 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:28:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 msgr2=0x7f30e006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 0x7f30e006ee40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0cc1e0 tx=0x7f30ec009040 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 msgr2=0x7f30fc15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009d90 tx=0x7f30f8004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 shutdown_connections 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f30e006c980 0x7f30e006ee40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f30fc0d26f0 0x7f30fc0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30fc0cf870 0x7f30fc0ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 --2- 172.21.15.62:0/4019279356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f30fc0b2290 0x7f30fc15b810 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 >> 172.21.15.62:0/4019279356 conn(0x7f30fc0d2ff0 msgr2=0x7f30fc06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 shutdown_connections 2024-04-26T22:28:18.261 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.260+0000 7f3103a64640 1 -- 172.21.15.62:0/4019279356 wait complete. 2024-04-26T22:28:18.271 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:28:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.375+0000 7f7f8700f640 1 Processor -- start 2024-04-26T22:28:18.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.375+0000 7f7f8700f640 1 -- start start 2024-04-26T22:28:18.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.375+0000 7f7f8700f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 0x7f7f8006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f8700f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f8015c830 con 0x7f7f800cbcd0 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f8700f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f8015d600 con 0x7f7f800cc840 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f8700f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8015d8e0 con 0x7f7f8015c450 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f7ffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7f800cbcd0 0x7f7f800cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53516/0 (socket says 172.21.15.62:53516) 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f7ffff640 1 -- 172.21.15.62:0/3849634504 learned_addr learned my addr 172.21.15.62:0/3849634504 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f84dab640 1 --2- 172.21.15.62:0/3849634504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 0x7f7f8006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451827459 0 0) 0x7f7f8015c830 con 0x7f7f800cbcd0 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f70003520 con 0x7f7f800cbcd0 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f84dab640 1 -- 172.21.15.62:0/3849634504 >> v1:172.21.15.174:6789/0 conn(0x7f7f800cc840 legacy=0x7f7f800729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422767483 0 0) 0x7f7f8015d600 con 0x7f7f800cc840 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f84dab640 1 -- 172.21.15.62:0/3849634504 >> v1:172.21.15.62:6789/0 conn(0x7f7f800cbcd0 legacy=0x7f7f800cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.377 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.376+0000 7f7f84dab640 1 -- 172.21.15.62:0/3849634504 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8006f830 con 0x7f7f8015c450 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.377+0000 7f7f84dab640 1 --2- 172.21.15.62:0/3849634504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 0x7f7f8006df90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f740097d0 tx=0x7f7f7402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f32fd671c6a9eab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.377+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74037070 con 0x7f7f8015c450 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.377+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74032440 con 0x7f7f8015c450 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.377+0000 7f7f7f7fe640 1 -- 172.21.15.62:0/3849634504 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f740327f0 con 0x7f7f8015c450 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 -- 172.21.15.62:0/3849634504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 msgr2=0x7f7f8006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 --2- 172.21.15.62:0/3849634504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 0x7f7f8006df90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f740097d0 tx=0x7f7f7402a180 comp rx=0 tx=0).stop 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 -- 172.21.15.62:0/3849634504 shutdown_connections 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 --2- 172.21.15.62:0/3849634504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f8015c450 0x7f7f8006df90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 -- 172.21.15.62:0/3849634504 >> 172.21.15.62:0/3849634504 conn(0x7f7f800d0d70 msgr2=0x7f7f800d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 -- 172.21.15.62:0/3849634504 shutdown_connections 2024-04-26T22:28:18.379 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.378+0000 7f7f8700f640 1 -- 172.21.15.62:0/3849634504 wait complete. 2024-04-26T22:28:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.379+0000 7f7f8700f640 1 Processor -- start 2024-04-26T22:28:18.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.379+0000 7f7f8700f640 1 -- start start 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f800cc840 0x7f7f800c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8015c450 0x7f7f800c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80073280 con 0x7f7f800cbcd0 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80073100 con 0x7f7f8015c450 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f8700f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7f80073400 con 0x7f7f800cc840 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38618/0 (socket says 172.21.15.62:38618) 2024-04-26T22:28:18.382 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 -- 172.21.15.62:0/4042008097 learned_addr learned my addr 172.21.15.62:0/4042008097 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f7ffff640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f800cc840 0x7f7f800c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f855ac640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8015c450 0x7f7f800c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 -- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f800cc840 msgr2=0x7f7f800c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f800cc840 0x7f7f800c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 -- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8015c450 msgr2=0x7f7f800c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8015c450 0x7f7f800c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 -- 172.21.15.62:0/4042008097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f800b3d30 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f84dab640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f7402a1c0 tx=0x7f7f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.380+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74037040 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.381+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f74042070 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.381+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74030900 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.381+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f800b4010 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.381+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f800b45d0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.382+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8006a360 con 0x7f7f800cbcd0 2024-04-26T22:28:18.383 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.382+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7f74030aa0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.382+0000 7f7f7d7fa640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 0x7f7f6406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.382+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f7f740ac5a0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.383+0000 7f7f7ffff640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 0x7f7f6406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.384 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.383+0000 7f7f7ffff640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 0x7f7f6406ee40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f801f8620 tx=0x7f7f6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.387+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f7407a9e0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.574+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f800b6cf0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.574+0000 7f7f7d7fa640 1 -- 172.21.15.62:0/4042008097 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) ==== 104+0+11 (secure 0 0 0) 0x7f7f7407a2e0 con 0x7f7f800cbcd0 2024-04-26T22:28:18.575 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 msgr2=0x7f7f6406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 0x7f7f6406ee40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7f801f8620 tx=0x7f7f6c009040 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 msgr2=0x7f7f80154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f7402a1c0 tx=0x7f7f740046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 shutdown_connections 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7f6406c980 0x7f7f6406ee40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8015c450 0x7f7f800c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7f800cc840 0x7f7f800c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 --2- 172.21.15.62:0/4042008097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7f800cbcd0 0x7f7f80154110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 >> 172.21.15.62:0/4042008097 conn(0x7f7f800d0d70 msgr2=0x7f7f800b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 shutdown_connections 2024-04-26T22:28:18.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.576+0000 7f7f8700f640 1 -- 172.21.15.62:0/4042008097 wait complete. 2024-04-26T22:28:18.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:28:18.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.691+0000 7f7b36a20640 1 Processor -- start 2024-04-26T22:28:18.691 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.691+0000 7f7b36a20640 1 -- start start 2024-04-26T22:28:18.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b3015f470 0x7f7b300d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b300d0d30 0x7f7b300d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7b300d1fe0 0x7f7b30162fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f7b300d1190 con 0x7f7b3015f470 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f7b300d2ed0 con 0x7f7b300d1fe0 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b36a20640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7b300d31b0 con 0x7f7b300d0d30 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b3015f470 0x7f7b300d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f7b3015f470 0x7f7b300d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38626/0 (socket says 172.21.15.62:38626) 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 -- 172.21.15.62:0/2737701273 learned_addr learned my addr 172.21.15.62:0/2737701273 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2f7fe640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b300d0d30 0x7f7b300d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b34fbd640 1 --2- 172.21.15.62:0/2737701273 >> v2:172.21.15.174:3300/0 conn(0x7f7b300d1fe0 0x7f7b30162fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 -- 172.21.15.62:0/2737701273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b300d0d30 msgr2=0x7f7b300d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b300d0d30 0x7f7b300d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 -- 172.21.15.62:0/2737701273 >> v2:172.21.15.174:3300/0 conn(0x7f7b300d1fe0 msgr2=0x7f7b30162fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/2737701273 >> v2:172.21.15.174:3300/0 conn(0x7f7b300d1fe0 0x7f7b30162fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.693 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.692+0000 7f7b2ffff640 1 -- 172.21.15.62:0/2737701273 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b300d3490 con 0x7f7b3015f470 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.693+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3015f470 0x7f7b300d07f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009870 tx=0x7f7b2002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ccbd2933fc0461 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.693+0000 7f7b2effd640 1 -- 172.21.15.62:0/2737701273 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20037070 con 0x7f7b3015f470 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.693+0000 7f7b2effd640 1 -- 172.21.15.62:0/2737701273 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b20032440 con 0x7f7b3015f470 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.693+0000 7f7b2effd640 1 -- 172.21.15.62:0/2737701273 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20032810 con 0x7f7b3015f470 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 -- 172.21.15.62:0/2737701273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3015f470 msgr2=0x7f7b300d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3015f470 0x7f7b300d07f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009870 tx=0x7f7b2002a210 comp rx=0 tx=0).stop 2024-04-26T22:28:18.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 -- 172.21.15.62:0/2737701273 shutdown_connections 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 --2- 172.21.15.62:0/2737701273 >> v2:172.21.15.174:3300/0 conn(0x7f7b300d1fe0 0x7f7b30162fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b300d0d30 0x7f7b300d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 --2- 172.21.15.62:0/2737701273 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b3015f470 0x7f7b300d07f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 -- 172.21.15.62:0/2737701273 >> 172.21.15.62:0/2737701273 conn(0x7f7b3009e510 msgr2=0x7f7b300d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 -- 172.21.15.62:0/2737701273 shutdown_connections 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.694+0000 7f7b36a20640 1 -- 172.21.15.62:0/2737701273 wait complete. 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.695+0000 7f7b36a20640 1 Processor -- start 2024-04-26T22:28:18.696 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.695+0000 7f7b36a20640 1 -- start start 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 0x7f7b301601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 0x7f7b300ba840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3015f850 con 0x7f7b300d0d30 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b300d3a50 con 0x7f7b300d1fe0 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b36a20640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7b300d3f70 con 0x7f7b3015f470 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.698 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38628/0 (socket says 172.21.15.62:38628) 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 -- 172.21.15.62:0/841070042 learned_addr learned my addr 172.21.15.62:0/841070042 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b34fbd640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 0x7f7b300ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2f7fe640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 0x7f7b301601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 -- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 msgr2=0x7f7b300ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 0x7f7b300ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 -- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 msgr2=0x7f7b301601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 0x7f7b301601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2ffff640 1 -- 172.21.15.62:0/841070042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b30160710 con 0x7f7b300d0d30 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b2f7fe640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 0x7f7b301601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.696+0000 7f7b34fbd640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 0x7f7b300ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b2ffff640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000c00 tx=0x7f7b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20037040 con 0x7f7b300d0d30 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b200304c0 con 0x7f7b300d0d30 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200308b0 con 0x7f7b300d0d30 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b300cd7c0 con 0x7f7b300d0d30 2024-04-26T22:28:18.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.697+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b300cdd80 con 0x7f7b300d0d30 2024-04-26T22:28:18.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.698+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b30069af0 con 0x7f7b300d0d30 2024-04-26T22:28:18.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.698+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7b20044020 con 0x7f7b300d0d30 2024-04-26T22:28:18.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.699+0000 7f7b2cff9640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 0x7f7b1c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:18.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.699+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f7b200abbd0 con 0x7f7b300d0d30 2024-04-26T22:28:18.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.699+0000 7f7b2f7fe640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 0x7f7b1c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:18.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.699+0000 7f7b2f7fe640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 0x7f7b1c06ee70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006fd0 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:18.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.703+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b30069af0 con 0x7f7b300d0d30 2024-04-26T22:28:18.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.887+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b3006a360 con 0x7f7b300d0d30 2024-04-26T22:28:18.888 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.888+0000 7f7b2cff9640 1 -- 172.21.15.62:0/841070042 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) ==== 100+0+11 (secure 0 0 0) 0x7f7b2004b020 con 0x7f7b300d0d30 2024-04-26T22:28:18.888 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 msgr2=0x7f7b1c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 0x7f7b1c06ee70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006fd0 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 msgr2=0x7f7b3015bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7b20000c00 tx=0x7f7b200046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 shutdown_connections 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b1c06c9b0 0x7f7b1c06ee70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b3015f470 0x7f7b300ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.890 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b300d1fe0 0x7f7b301601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 --2- 172.21.15.62:0/841070042 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b300d0d30 0x7f7b3015bd40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 >> 172.21.15.62:0/841070042 conn(0x7f7b3009e510 msgr2=0x7f7b300cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 shutdown_connections 2024-04-26T22:28:18.891 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:18.890+0000 7f7b36a20640 1 -- 172.21.15.62:0/841070042 wait complete. 2024-04-26T22:28:18.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:28:19.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.002+0000 7f941fb46640 1 Processor -- start 2024-04-26T22:28:19.003 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 -- start start 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f94180b4ba0 0x7f94180b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94180d0210 con 0x7f9418162b40 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f94180d41c0 con 0x7f94180b4ba0 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.003+0000 7f941fb46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f94180d04f0 con 0x7f94180d3820 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941e0e3640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f94180b4ba0 0x7f94180b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941d8e2640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9418162b40 0x7f94180d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53528/0 (socket says 172.21.15.62:53528) 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941d8e2640 1 -- 172.21.15.62:0/1177995841 learned_addr learned my addr 172.21.15.62:0/1177995841 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941d0e1640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941e0e3640 1 -- 172.21.15.62:0/1177995841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 msgr2=0x7f94180d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:19.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941e0e3640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941e0e3640 1 -- 172.21.15.62:0/1177995841 >> v1:172.21.15.62:6789/0 conn(0x7f9418162b40 legacy=0x7f94180d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941e0e3640 1 -- 172.21.15.62:0/1177995841 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94180d07d0 con 0x7f94180b4ba0 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941c8e0640 1 -- 172.21.15.62:0/1177995841 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736868591 0 0) 0x7f94180d0210 con 0x7f9418162b40 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.004+0000 7f941d0e1640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941e0e3640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94180b4ba0 0x7f94180b5030 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94100097a0 tx=0x7f941002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fc5fe9480bb67f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941c8e0640 1 -- 172.21.15.62:0/1177995841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410037070 con 0x7f94180b4ba0 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941c8e0640 1 -- 172.21.15.62:0/1177995841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410032440 con 0x7f94180b4ba0 2024-04-26T22:28:19.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941c8e0640 1 -- 172.21.15.62:0/1177995841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410032810 con 0x7f94180b4ba0 2024-04-26T22:28:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 -- 172.21.15.62:0/1177995841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94180b4ba0 msgr2=0x7f94180b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94180b4ba0 0x7f94180b5030 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f94100097a0 tx=0x7f941002a180 comp rx=0 tx=0).stop 2024-04-26T22:28:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 -- 172.21.15.62:0/1177995841 shutdown_connections 2024-04-26T22:28:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94180b4ba0 0x7f94180b5030 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 --2- 172.21.15.62:0/1177995841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 -- 172.21.15.62:0/1177995841 >> 172.21.15.62:0/1177995841 conn(0x7f941809e550 msgr2=0x7f94180d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 -- 172.21.15.62:0/1177995841 shutdown_connections 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.005+0000 7f941fb46640 1 -- 172.21.15.62:0/1177995841 wait complete. 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 Processor -- start 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 -- start start 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 0x7f941806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f94180d12b0 con 0x7f9418162b40 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94180d0d90 con 0x7f94180b4ba0 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941fb46640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f94180cfe30 con 0x7f94180d3820 2024-04-26T22:28:19.010 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941e0e3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.007+0000 7f941e0e3640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38648/0 (socket says 172.21.15.62:38648) 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 -- 172.21.15.62:0/2779473566 learned_addr learned my addr 172.21.15.62:0/2779473566 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941d0e1640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941d8e2640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 0x7f941806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 -- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 msgr2=0x7f94180ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 -- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 msgr2=0x7f941806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 0x7f941806d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 -- 172.21.15.62:0/2779473566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94180cc3b0 con 0x7f9418162b40 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941d8e2640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 0x7f941806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941e0e3640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9410035040 tx=0x7f94100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410037040 con 0x7f9418162b40 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410042070 con 0x7f9418162b40 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94180cc680 con 0x7f9418162b40 2024-04-26T22:28:19.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410030b50 con 0x7f9418162b40 2024-04-26T22:28:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.008+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94180cf090 con 0x7f9418162b40 2024-04-26T22:28:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.009+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941806a360 con 0x7f9418162b40 2024-04-26T22:28:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.010+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9410030cf0 con 0x7f9418162b40 2024-04-26T22:28:19.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.010+0000 7f94167fc640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 0x7f940406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.010+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f94100ac7c0 con 0x7f9418162b40 2024-04-26T22:28:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.010+0000 7f941d8e2640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 0x7f940406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.011+0000 7f941d8e2640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 0x7f940406ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9408007fd0 tx=0x7f9408009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.014+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94180cc680 con 0x7f9418162b40 2024-04-26T22:28:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.210+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94180d3120 con 0x7f9418162b40 2024-04-26T22:28:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.211+0000 7f94167fc640 1 -- 172.21.15.62:0/2779473566 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) ==== 80+0+0 (secure 0 0 0) 0x7f9410032e20 con 0x7f9418162b40 2024-04-26T22:28:19.211 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.212+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 msgr2=0x7f940406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.212+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 0x7f940406ee40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9408007fd0 tx=0x7f9408009040 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.212+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 msgr2=0x7f94180b9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.212+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9410035040 tx=0x7f94100046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 shutdown_connections 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f940406c980 0x7f940406ee40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9418162b40 0x7f94180b9420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f94180d3820 0x7f94180ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 --2- 172.21.15.62:0/2779473566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94180b4ba0 0x7f941806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 >> 172.21.15.62:0/2779473566 conn(0x7f941809e550 msgr2=0x7f94180b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:19.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 shutdown_connections 2024-04-26T22:28:19.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.213+0000 7f941fb46640 1 -- 172.21.15.62:0/2779473566 wait complete. 2024-04-26T22:28:19.223 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T22:28:19.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:19.224 DEBUG:teuthology.orchestra.run.smithi062:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:19.255 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi062.front.sepia.ceph.com 2024-04-26T22:28:19.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:19.256 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T22:28:19.312 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T22:28:19.312 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:19.312 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:19.331 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:19.331 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:19.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.490+0000 7f48161af640 1 Processor -- start 2024-04-26T22:28:19.491 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.490+0000 7f48161af640 1 -- start start 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.491+0000 7f48161af640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4810162b40 0x7f48100d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.491+0000 7f48161af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.491+0000 7f48161af640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48100d0210 con 0x7f48100d3820 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.491+0000 7f48161af640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f48100d04f0 con 0x7f4810162b40 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.491+0000 7f48161af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f48100d07d0 con 0x7f48100d3c80 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4810162b40 0x7f48100d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480effd640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f48100d3820 0x7f48100d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:43302/0 (socket says 172.21.15.62:43302) 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4810162b40 0x7f48100d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:52626/0 (socket says 172.21.15.62:52626) 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480ffff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:48500/0 (socket says 172.21.15.62:48500) 2024-04-26T22:28:19.492 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480effd640 1 -- 172.21.15.62:0/4236904418 learned_addr learned my addr 172.21.15.62:0/4236904418 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 -- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 msgr2=0x7f48100d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 -- 172.21.15.62:0/4236904418 >> v1:172.21.15.174:6789/0 conn(0x7f48100d3820 legacy=0x7f48100d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480f7fe640 1 -- 172.21.15.62:0/4236904418 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100d0ab0 con 0x7f4810162b40 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.492+0000 7f480ffff640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:19.494 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.493+0000 7f480f7fe640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4810162b40 0x7f48100d5bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4800009d90 tx=0x7f480002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132dcf76fc6832b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.493+0000 7f480e7fc640 1 -- 172.21.15.62:0/4236904418 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800007740 con 0x7f4810162b40 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.493+0000 7f480e7fc640 1 -- 172.21.15.62:0/4236904418 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48000078e0 con 0x7f4810162b40 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.493+0000 7f480e7fc640 1 -- 172.21.15.62:0/4236904418 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800007cb0 con 0x7f4810162b40 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 -- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4810162b40 msgr2=0x7f48100d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4810162b40 0x7f48100d5bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4800009d90 tx=0x7f480002a6f0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 -- 172.21.15.62:0/4236904418 shutdown_connections 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f48100d3c80 0x7f48100d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 --2- 172.21.15.62:0/4236904418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4810162b40 0x7f48100d5bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 -- 172.21.15.62:0/4236904418 >> 172.21.15.62:0/4236904418 conn(0x7f481009e550 msgr2=0x7f48100d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 -- 172.21.15.62:0/4236904418 shutdown_connections 2024-04-26T22:28:19.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.494+0000 7f48161af640 1 -- 172.21.15.62:0/4236904418 wait complete. 2024-04-26T22:28:19.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.495+0000 7f48161af640 1 Processor -- start 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.495+0000 7f48161af640 1 -- start start 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48100d3820 0x7f481015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4810162b40 0x7f48101f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f48100b5bb0 con 0x7f48100d3c80 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48100d4110 con 0x7f48100d3820 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f48161af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f48100d6110 con 0x7f4810162b40 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38662/0 (socket says 172.21.15.62:38662) 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 -- 172.21.15.62:0/3972905844 learned_addr learned my addr 172.21.15.62:0/3972905844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480f7fe640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48100d3820 0x7f481015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480ffff640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4810162b40 0x7f48101f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 -- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4810162b40 msgr2=0x7f48101f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4810162b40 0x7f48101f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 -- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48100d3820 msgr2=0x7f481015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48100d3820 0x7f481015eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 -- 172.21.15.62:0/3972905844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100ceb10 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.496+0000 7f480effd640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f480400c3f0 tx=0x7f480400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.497+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804017070 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.497+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4804005ce0 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.497+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48100b8270 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.497+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48040072f0 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.497+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4810161190 con 0x7f48100d3c80 2024-04-26T22:28:19.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.498+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481006a360 con 0x7f48100d3c80 2024-04-26T22:28:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.498+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f480401a070 con 0x7f48100d3c80 2024-04-26T22:28:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.498+0000 7f47f7fff640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 0x7f47fc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.499+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f480408ad00 con 0x7f48100d3c80 2024-04-26T22:28:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.499+0000 7f480f7fe640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 0x7f47fc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.499+0000 7f480f7fe640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 0x7f47fc06ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.503+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f480408e030 con 0x7f48100d3c80 2024-04-26T22:28:19.690 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.690+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48100d2960 con 0x7f48100d3c80 2024-04-26T22:28:19.692 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.691+0000 7f47f7fff640 1 -- 172.21.15.62:0/3972905844 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7f4804058b80 con 0x7f48100d3c80 2024-04-26T22:28:19.692 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:19.692 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 msgr2=0x7f47fc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 0x7f47fc06ee40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4800034000 tx=0x7f480002f000 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 msgr2=0x7f4810160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f480400c3f0 tx=0x7f480400c7f0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 shutdown_connections 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f47fc06c980 0x7f47fc06ee40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4810162b40 0x7f48101f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f48100d3c80 0x7f4810160c50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 --2- 172.21.15.62:0/3972905844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48100d3820 0x7f481015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 >> 172.21.15.62:0/3972905844 conn(0x7f481009e550 msgr2=0x7f48100d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:19.694 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.693+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 shutdown_connections 2024-04-26T22:28:19.695 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.694+0000 7f48161af640 1 -- 172.21.15.62:0/3972905844 wait complete. 2024-04-26T22:28:19.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:28:00.329554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:28:00.858734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:28:19.705 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:28:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.809+0000 7f4af0ae1640 1 Processor -- start 2024-04-26T22:28:19.809 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.809+0000 7f4af0ae1640 1 -- start start 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4af0ae1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4aec0b2930 0x7f4aec0d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4af0ae1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec0cd640 con 0x7f4aec0b2290 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4af0ae1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4aec0cd920 con 0x7f4aec0cff80 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4af0ae1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec0cdc00 con 0x7f4aec0b2930 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4aea59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4aec0b2290 0x7f4aec0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53530/0 (socket says 172.21.15.62:53530) 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4aea59c640 1 -- 172.21.15.62:0/400277203 learned_addr learned my addr 172.21.15.62:0/400277203 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810980227 0 0) 0x7f4aec0cd640 con 0x7f4aec0b2290 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ad8003520 con 0x7f4aec0b2290 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> v2:172.21.15.174:3300/0 conn(0x7f4aec0b2930 0x7f4aec0d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.812 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2432936644 0 0) 0x7f4ad8003520 con 0x7f4aec0b2290 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 >> v2:172.21.15.174:3300/0 conn(0x7f4aec0b2930 msgr2=0x7f4aec0d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 --2- 172.21.15.62:0/400277203 >> v2:172.21.15.174:3300/0 conn(0x7f4aec0b2930 0x7f4aec0d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 >> v1:172.21.15.174:6789/0 conn(0x7f4aec0cff80 legacy=0x7f4aec0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec0cdee0 con 0x7f4aec0b2290 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f4adc001670 con 0x7f4aec0b2290 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> v2:172.21.15.174:3300/0 conn(0x7f4aec0b2930 0x7f4aec0d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 >> v1:172.21.15.62:6789/0 conn(0x7f4aec0b2290 legacy=0x7f4aec0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ad80039b0 0x7f4ad8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8004800 0x7f4ad8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 0x7f4ad800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0cdc00 con 0x7f4ad80051c0 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad800c110 con 0x7f4ad8004800 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.810+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad800c310 con 0x7f4ad80039b0 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 0x7f4ad800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4ae3fff640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8004800 0x7f4ad8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.813 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aea59c640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ad80039b0 0x7f4ad8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 -- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ad80039b0 msgr2=0x7f4ad8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ad80039b0 0x7f4ad8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 -- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8004800 msgr2=0x7f4ad8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad8004800 0x7f4ad8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 -- 172.21.15.62:0/400277203 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad8003520 con 0x7f4ad80051c0 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.811+0000 7f4aead9d640 1 -- 172.21.15.62:0/400277203 reap_dead start 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4aead9d640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 0x7f4ad800b9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40073e0 tx=0x7f4ad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869ec826aa53e442 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4009c80 con 0x7f4ad80051c0 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae402c6c0 con 0x7f4ad80051c0 2024-04-26T22:28:19.814 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4ae9d9b640 1 -- 172.21.15.62:0/400277203 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402cab0 con 0x7f4ad80051c0 2024-04-26T22:28:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 -- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 msgr2=0x7f4ad800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 0x7f4ad800b9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40073e0 tx=0x7f4ad8005830 comp rx=0 tx=0).stop 2024-04-26T22:28:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 -- 172.21.15.62:0/400277203 shutdown_connections 2024-04-26T22:28:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/400277203 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4ad80051c0 0x7f4ad800b9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.817 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 -- 172.21.15.62:0/400277203 >> 172.21.15.62:0/400277203 conn(0x7f4aec0d2ff0 msgr2=0x7f4aec0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 -- 172.21.15.62:0/400277203 shutdown_connections 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.812+0000 7f4af0ae1640 1 -- 172.21.15.62:0/400277203 wait complete. 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 Processor -- start 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 -- start start 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 0x7f4aec0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aec1536b0 0x7f4aec0c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0cf870 con 0x7f4aec0cbc70 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b2670 con 0x7f4ad80051c0 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.814+0000 7f4af0ae1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec0cf9f0 con 0x7f4aec1536b0 2024-04-26T22:28:19.818 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38676/0 (socket says 172.21.15.62:38676) 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 -- 172.21.15.62:0/824823776 learned_addr learned my addr 172.21.15.62:0/824823776 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aea59c640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 0x7f4aec0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4ae3fff640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aec1536b0 0x7f4aec0c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 -- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aec1536b0 msgr2=0x7f4aec0c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aec1536b0 0x7f4aec0c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 -- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 msgr2=0x7f4aec0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 0x7f4aec0c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 -- 172.21.15.62:0/824823776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec0c7d80 con 0x7f4aec0cbc70 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aea59c640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 0x7f4aec0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4aead9d640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4ae402b160 tx=0x7f4ae4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4009c80 con 0x7f4aec0cbc70 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae402da20 con 0x7f4aec0cbc70 2024-04-26T22:28:19.819 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae402de10 con 0x7f4aec0cbc70 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec0c8060 con 0x7f4aec0cbc70 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.815+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec0b6c40 con 0x7f4aec0cbc70 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.816+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec0c8580 con 0x7f4aec0cbc70 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.817+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4ae4036400 con 0x7f4aec0cbc70 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.817+0000 7f4ae2ffd640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 0x7f4acc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:19.820 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.817+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f4ae40ac120 con 0x7f4aec0cbc70 2024-04-26T22:28:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.818+0000 7f4aea59c640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 0x7f4acc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.818+0000 7f4aea59c640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 0x7f4acc06ee70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0d3a10 tx=0x7f4adc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:19.821 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:19.822+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4aec0c8580 con 0x7f4aec0cbc70 2024-04-26T22:28:20.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.004+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4aec06a360 con 0x7f4aec0cbc70 2024-04-26T22:28:20.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.005+0000 7f4ae2ffd640 1 -- 172.21.15.62:0/824823776 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f4ae4032030 con 0x7f4aec0cbc70 2024-04-26T22:28:20.005 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.006+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 msgr2=0x7f4acc06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.006+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 0x7f4acc06ee70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0d3a10 tx=0x7f4adc009040 comp rx=0 tx=0).stop 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.006+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 msgr2=0x7f4aec0cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.006+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4ae402b160 tx=0x7f4ae4004ab0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 shutdown_connections 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4acc06c9b0 0x7f4acc06ee70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aec0cbc70 0x7f4aec0cc120 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aec1536b0 0x7f4aec0c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 --2- 172.21.15.62:0/824823776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad80051c0 0x7f4aec0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 >> 172.21.15.62:0/824823776 conn(0x7f4aec0d2ff0 msgr2=0x7f4aec0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 shutdown_connections 2024-04-26T22:28:20.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.007+0000 7f4af0ae1640 1 -- 172.21.15.62:0/824823776 wait complete. 2024-04-26T22:28:20.018 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:28:20.121 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.121+0000 7f7b53bf0640 1 Processor -- start 2024-04-26T22:28:20.122 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.121+0000 7f7b53bf0640 1 -- start start 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b53bf0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f7b4c0d2320 0x7f7b4c0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b53bf0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c161510 con 0x7f7b4c1610c0 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b53bf0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c0d0930 con 0x7f7b4c160800 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b53bf0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c0d0c00 con 0x7f7b4c0d2320 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b5118b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f7b4c1610c0 0x7f7b4c0d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53536/0 (socket says 172.21.15.62:53536) 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.122+0000 7f7b5118b640 1 -- 172.21.15.62:0/3617961285 learned_addr learned my addr 172.21.15.62:0/3617961285 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055881292 0 0) 0x7f7b4c161510 con 0x7f7b4c1610c0 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 --2- 172.21.15.62:0/3617961285 >> v2:172.21.15.174:3300/0 conn(0x7f7b4c0d2320 0x7f7b4c0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.125 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b34003520 con 0x7f7b4c1610c0 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1345148317 0 0) 0x7f7b34003520 con 0x7f7b4c1610c0 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 >> v2:172.21.15.174:3300/0 conn(0x7f7b4c0d2320 msgr2=0x7f7b4c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 --2- 172.21.15.62:0/3617961285 >> v2:172.21.15.174:3300/0 conn(0x7f7b4c0d2320 0x7f7b4c0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 >> v1:172.21.15.174:6789/0 conn(0x7f7b4c160800 legacy=0x7f7b4c0d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c0d0ee0 con 0x7f7b4c1610c0 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f7b40001670 con 0x7f7b4c1610c0 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 >> v1:172.21.15.62:6789/0 conn(0x7f7b4c1610c0 legacy=0x7f7b4c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b34003a00 0x7f7b34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0d0c00 con 0x7f7b34005210 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400c160 con 0x7f7b34003a00 2024-04-26T22:28:20.126 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3400c360 con 0x7f7b34004800 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5198c640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b34003a00 0x7f7b34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5118b640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 -- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b34004800 msgr2=0x7f7b34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b34004800 0x7f7b34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 -- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b34003a00 msgr2=0x7f7b34003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b34003a00 0x7f7b34003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.123+0000 7f7b5218d640 1 -- 172.21.15.62:0/3617961285 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b34003520 con 0x7f7b34005210 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.124+0000 7f7b5218d640 1 -- 172.21.15.62:0/3617961285 reap_dead start 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.124+0000 7f7b5218d640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b3400b9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009f90 tx=0x7f7b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8f39eb580c9a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.124+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c035020 con 0x7f7b34005210 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.124+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3c003b90 con 0x7f7b34005210 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.124+0000 7f7b5098a640 1 -- 172.21.15.62:0/3617961285 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c009c10 con 0x7f7b34005210 2024-04-26T22:28:20.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 -- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 msgr2=0x7f7b3400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b3400b9c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009f90 tx=0x7f7b34005830 comp rx=0 tx=0).stop 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 -- 172.21.15.62:0/3617961285 shutdown_connections 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/3617961285 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b3400b9c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 -- 172.21.15.62:0/3617961285 >> 172.21.15.62:0/3617961285 conn(0x7f7b4c09e550 msgr2=0x7f7b4c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 -- 172.21.15.62:0/3617961285 shutdown_connections 2024-04-26T22:28:20.128 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.125+0000 7f7b53bf0640 1 -- 172.21.15.62:0/3617961285 wait complete. 2024-04-26T22:28:20.130 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.126+0000 7f7b53bf0640 1 Processor -- start 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.126+0000 7f7b53bf0640 1 -- start start 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 0x7f7b4c14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b4c14e5f0 0x7f7b4c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0d30f0 con 0x7f7b34005210 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0d2e10 con 0x7f7b4c14e5f0 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b6300 con 0x7f7b4c0ce590 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38692/0 (socket says 172.21.15.62:38692) 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 -- 172.21.15.62:0/4000991661 learned_addr learned my addr 172.21.15.62:0/4000991661 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5218d640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b4c14e5f0 0x7f7b4c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5118b640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 0x7f7b4c14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 -- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 msgr2=0x7f7b4c14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 0x7f7b4c14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 -- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b4c14e5f0 msgr2=0x7f7b4c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b4c14e5f0 0x7f7b4c157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 -- 172.21.15.62:0/4000991661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c0cf0b0 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5118b640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 0x7f7b4c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b5198c640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e410 tx=0x7f7b4400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44019070 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b440052e0 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c0ceaf0 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.127+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440037f0 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.128+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c0b8d40 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.129+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c06a360 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.129+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f7b4401c070 con 0x7f7b34005210 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.129+0000 7f7b4a7fc640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 0x7f7b3806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.129+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f7b4408b120 con 0x7f7b34005210 2024-04-26T22:28:20.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.129+0000 7f7b5118b640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 0x7f7b3806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.130+0000 7f7b5118b640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 0x7f7b3806ee70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b40001eb0 tx=0x7f7b40000f50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.133 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.134+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b44058f10 con 0x7f7b34005210 2024-04-26T22:28:20.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.318+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b4c0d4530 con 0x7f7b34005210 2024-04-26T22:28:20.318 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.318+0000 7f7b4a7fc640 1 -- 172.21.15.62:0/4000991661 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) ==== 104+0+11 (secure 0 0 0) 0x7f7b4405d210 con 0x7f7b34005210 2024-04-26T22:28:20.319 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 msgr2=0x7f7b3806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 0x7f7b3806ee70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b40001eb0 tx=0x7f7b40000f50 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 msgr2=0x7f7b4c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e410 tx=0x7f7b4400e8d0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 shutdown_connections 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f7b3806c9b0 0x7f7b3806ee70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b4c14e5f0 0x7f7b4c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7b4c0ce590 0x7f7b4c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 --2- 172.21.15.62:0/4000991661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7b34005210 0x7f7b4c0ce050 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.321 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 >> 172.21.15.62:0/4000991661 conn(0x7f7b4c09e550 msgr2=0x7f7b4c0b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 shutdown_connections 2024-04-26T22:28:20.322 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.320+0000 7f7b53bf0640 1 -- 172.21.15.62:0/4000991661 wait complete. 2024-04-26T22:28:20.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:28:20.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.435+0000 7f218da50640 1 Processor -- start 2024-04-26T22:28:20.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.435+0000 7f218da50640 1 -- start start 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f218da50640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f218815f470 0x7f21880d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f218da50640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21880d21b0 0x7f21880b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f218da50640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21880d2ed0 con 0x7f21880d0d30 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f218da50640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f21880d1190 con 0x7f21880d21b0 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f218da50640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f21880d31b0 con 0x7f218815f470 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21880d21b0 0x7f21880b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f21880d21b0 0x7f21880b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38706/0 (socket says 172.21.15.62:38706) 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 -- 172.21.15.62:0/2619469266 learned_addr learned my addr 172.21.15.62:0/2619469266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f2186ffd640 1 --2- 172.21.15.62:0/2619469266 >> v2:172.21.15.174:3300/0 conn(0x7f218815f470 0x7f21880d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 -- 172.21.15.62:0/2619469266 >> v2:172.21.15.174:3300/0 conn(0x7f218815f470 msgr2=0x7f21880d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:20.437 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 --2- 172.21.15.62:0/2619469266 >> v2:172.21.15.174:3300/0 conn(0x7f218815f470 0x7f21880d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.436+0000 7f21877fe640 1 -- 172.21.15.62:0/2619469266 >> v1:172.21.15.174:6789/0 conn(0x7f21880d0d30 legacy=0x7f21880d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.438 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.437+0000 7f21877fe640 1 -- 172.21.15.62:0/2619469266 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21880d3490 con 0x7f21880d21b0 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.437+0000 7f21877fe640 1 --2- 172.21.15.62:0/2619469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d21b0 0x7f21880b7fb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f21880d61a0 tx=0x7f217802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfef69399e2cb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.437+0000 7f2185ffb640 1 -- 172.21.15.62:0/2619469266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178036070 con 0x7f21880d21b0 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.437+0000 7f2185ffb640 1 -- 172.21.15.62:0/2619469266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178031600 con 0x7f21880d21b0 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.437+0000 7f2185ffb640 1 -- 172.21.15.62:0/2619469266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21780319d0 con 0x7f21880d21b0 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 -- 172.21.15.62:0/2619469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d21b0 msgr2=0x7f21880b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.439 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 --2- 172.21.15.62:0/2619469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d21b0 0x7f21880b7fb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f21880d61a0 tx=0x7f217802a730 comp rx=0 tx=0).stop 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 -- 172.21.15.62:0/2619469266 shutdown_connections 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 --2- 172.21.15.62:0/2619469266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d21b0 0x7f21880b7fb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 --2- 172.21.15.62:0/2619469266 >> v2:172.21.15.174:3300/0 conn(0x7f218815f470 0x7f21880d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 -- 172.21.15.62:0/2619469266 >> 172.21.15.62:0/2619469266 conn(0x7f218809e510 msgr2=0x7f21880d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 -- 172.21.15.62:0/2619469266 shutdown_connections 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.438+0000 7f218da50640 1 -- 172.21.15.62:0/2619469266 wait complete. 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.439+0000 7f218da50640 1 Processor -- start 2024-04-26T22:28:20.440 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.439+0000 7f218da50640 1 -- start start 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21880d21b0 0x7f218806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f218815f470 0x7f21880cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f218815f850 con 0x7f21880d0d30 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21880d3a50 con 0x7f218815f470 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f218da50640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f21880d3f70 con 0x7f21880d21b0 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f2186ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.442 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f2186ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38712/0 (socket says 172.21.15.62:38712) 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f2186ffd640 1 -- 172.21.15.62:0/4024730807 learned_addr learned my addr 172.21.15.62:0/4024730807 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f21877fe640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f218815f470 0x7f21880cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.440+0000 7f21867fc640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21880d21b0 0x7f218806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 -- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21880d21b0 msgr2=0x7f218806dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21880d21b0 0x7f218806dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 -- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f218815f470 msgr2=0x7f21880cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f218815f470 0x7f21880cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 -- 172.21.15.62:0/4024730807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21880cc610 con 0x7f21880d0d30 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f2186ffd640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f218000e870 tx=0x7f218000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180017070 con 0x7f21880d0d30 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218000c8e0 con 0x7f21880d0d30 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21880cc8f0 con 0x7f21880d0d30 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000ccd0 con 0x7f21880d0d30 2024-04-26T22:28:20.443 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.441+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21880cceb0 con 0x7f21880d0d30 2024-04-26T22:28:20.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.442+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218806a360 con 0x7f21880d0d30 2024-04-26T22:28:20.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.443+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f218001a070 con 0x7f21880d0d30 2024-04-26T22:28:20.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.443+0000 7f217ffff640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 0x7f216806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.444 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.443+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f218008bc10 con 0x7f21880d0d30 2024-04-26T22:28:20.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.443+0000 7f21867fc640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 0x7f216806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.445 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.444+0000 7f21867fc640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 0x7f216806ee70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f218815fa40 tx=0x7f2170009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.447 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.448+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2180059a90 con 0x7f21880d0d30 2024-04-26T22:28:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.635+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f21880d5870 con 0x7f21880d0d30 2024-04-26T22:28:20.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.635+0000 7f217ffff640 1 -- 172.21.15.62:0/4024730807 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) ==== 100+0+11 (secure 0 0 0) 0x7f218005dd90 con 0x7f21880d0d30 2024-04-26T22:28:20.636 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:28:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 msgr2=0x7f216806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 0x7f216806ee70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f218815fa40 tx=0x7f2170009040 comp rx=0 tx=0).stop 2024-04-26T22:28:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 msgr2=0x7f218814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f218000e870 tx=0x7f218000ed30 comp rx=0 tx=0).stop 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 shutdown_connections 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f216806c9b0 0x7f216806ee70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f218815f470 0x7f21880cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f21880d21b0 0x7f218806dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 --2- 172.21.15.62:0/4024730807 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21880d0d30 0x7f218814b430 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.637+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 >> 172.21.15.62:0/4024730807 conn(0x7f218809e510 msgr2=0x7f218806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.638+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 shutdown_connections 2024-04-26T22:28:20.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.638+0000 7f218da50640 1 -- 172.21.15.62:0/4024730807 wait complete. 2024-04-26T22:28:20.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T22:28:20.750 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.750+0000 7f36adc37640 1 Processor -- start 2024-04-26T22:28:20.751 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.750+0000 7f36adc37640 1 -- start start 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36adc37640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f36a80d22e0 0x7f36a80b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36adc37640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f36a815f760 0x7f36a80b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36adc37640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a80d3de0 con 0x7f36a815f380 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36adc37640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f36a80d43a0 con 0x7f36a815f760 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36adc37640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f36a80d40c0 con 0x7f36a80d22e0 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a7fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f36a815f760 0x7f36a80b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a77fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f36a815f380 0x7f36a80d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53552/0 (socket says 172.21.15.62:53552) 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a77fe640 1 -- 172.21.15.62:0/2159547979 learned_addr learned my addr 172.21.15.62:0/2159547979 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a6ffd640 1 --2- 172.21.15.62:0/2159547979 >> v2:172.21.15.174:3300/0 conn(0x7f36a80d22e0 0x7f36a80b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a7fff640 1 -- 172.21.15.62:0/2159547979 >> v2:172.21.15.174:3300/0 conn(0x7f36a80d22e0 msgr2=0x7f36a80b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a7fff640 1 --2- 172.21.15.62:0/2159547979 >> v2:172.21.15.174:3300/0 conn(0x7f36a80d22e0 0x7f36a80b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a7fff640 1 -- 172.21.15.62:0/2159547979 >> v1:172.21.15.62:6789/0 conn(0x7f36a815f380 legacy=0x7f36a80d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a7fff640 1 -- 172.21.15.62:0/2159547979 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a80b8280 con 0x7f36a815f760 2024-04-26T22:28:20.753 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.751+0000 7f36a67fc640 1 -- 172.21.15.62:0/2159547979 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624161398 0 0) 0x7f36a80d3de0 con 0x7f36a815f380 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.752+0000 7f36a7fff640 1 --2- 172.21.15.62:0/2159547979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a815f760 0x7f36a80b8fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009d90 tx=0x7f36a002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a049cc8218fe552f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.752+0000 7f36a67fc640 1 -- 172.21.15.62:0/2159547979 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0007740 con 0x7f36a815f760 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.752+0000 7f36a67fc640 1 -- 172.21.15.62:0/2159547979 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a00078e0 con 0x7f36a815f760 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.752+0000 7f36a67fc640 1 -- 172.21.15.62:0/2159547979 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0007cb0 con 0x7f36a815f760 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 -- 172.21.15.62:0/2159547979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a815f760 msgr2=0x7f36a80b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 --2- 172.21.15.62:0/2159547979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a815f760 0x7f36a80b8fc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009d90 tx=0x7f36a002a730 comp rx=0 tx=0).stop 2024-04-26T22:28:20.754 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 -- 172.21.15.62:0/2159547979 shutdown_connections 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 --2- 172.21.15.62:0/2159547979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a815f760 0x7f36a80b8fc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 --2- 172.21.15.62:0/2159547979 >> v2:172.21.15.174:3300/0 conn(0x7f36a80d22e0 0x7f36a80b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 -- 172.21.15.62:0/2159547979 >> 172.21.15.62:0/2159547979 conn(0x7f36a809e560 msgr2=0x7f36a80d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 -- 172.21.15.62:0/2159547979 shutdown_connections 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.753+0000 7f36adc37640 1 -- 172.21.15.62:0/2159547979 wait complete. 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.754+0000 7f36adc37640 1 Processor -- start 2024-04-26T22:28:20.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.754+0000 7f36adc37640 1 -- start start 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36a815f380 0x7f36a815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 0x7f36a815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36a80d1280 con 0x7f36a80d22e0 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36a80d1100 con 0x7f36a815f760 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36adc37640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f36a80d5b50 con 0x7f36a815f380 2024-04-26T22:28:20.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38726/0 (socket says 172.21.15.62:38726) 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 -- 172.21.15.62:0/335585789 learned_addr learned my addr 172.21.15.62:0/335585789 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a6ffd640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36a815f380 0x7f36a815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a7fff640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 0x7f36a815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 -- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36a815f380 msgr2=0x7f36a815d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36a815f380 0x7f36a815d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 -- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 msgr2=0x7f36a815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 0x7f36a815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a77fe640 1 -- 172.21.15.62:0/335585789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a81f9350 con 0x7f36a80d22e0 2024-04-26T22:28:20.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.755+0000 7f36a7fff640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 0x7f36a815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f36a77fe640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f369800c4d0 tx=0x7f369800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698018070 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698007500 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a81f5260 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698004a20 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.756+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81f5820 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.757+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a806a360 con 0x7f36a80d22e0 2024-04-26T22:28:20.759 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.757+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f369801b070 con 0x7f36a80d22e0 2024-04-26T22:28:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.758+0000 7f368ffff640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 0x7f369406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.758+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f369808aed0 con 0x7f36a80d22e0 2024-04-26T22:28:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.758+0000 7f36a6ffd640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 0x7f369406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:20.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.758+0000 7f36a6ffd640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 0x7f369406ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36a81593f0 tx=0x7f369c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:20.762 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.762+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3698059310 con 0x7f36a80d22e0 2024-04-26T22:28:20.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.958+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36a806d330 con 0x7f36a80d22e0 2024-04-26T22:28:20.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.958+0000 7f368ffff640 1 -- 172.21.15.62:0/335585789 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v31) ==== 80+0+0 (secure 0 0 0) 0x7f3698058c10 con 0x7f36a80d22e0 2024-04-26T22:28:20.958 INFO:teuthology.orchestra.run.smithi062.stderr:listed 0 entries 2024-04-26T22:28:20.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 msgr2=0x7f369406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 0x7f369406ee40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36a81593f0 tx=0x7f369c008040 comp rx=0 tx=0).stop 2024-04-26T22:28:20.960 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 msgr2=0x7f36a8155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f369800c4d0 tx=0x7f369800c990 comp rx=0 tx=0).stop 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 shutdown_connections 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f369406c980 0x7f369406ee40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a815f760 0x7f36a815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36a815f380 0x7f36a815d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 --2- 172.21.15.62:0/335585789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f36a80d22e0 0x7f36a8155a50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 >> 172.21.15.62:0/335585789 conn(0x7f36a809e560 msgr2=0x7f36a80b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 shutdown_connections 2024-04-26T22:28:20.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:20.960+0000 7f36adc37640 1 -- 172.21.15.62:0/335585789 wait complete. 2024-04-26T22:28:20.971 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T22:28:20.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:20.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:21.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T22:28:21.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:21.003 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T22:28:21.061 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T22:28:21.061 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:21.061 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:21.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056113,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T22:28:21.078 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T22:28:21.078 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:21.078 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:21.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17182 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T22:28:21.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.218+0000 7f4924de4640 1 Processor -- start 2024-04-26T22:28:21.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 -- start start 2024-04-26T22:28:21.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4920162b40 0x7f49200d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f49200d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49200d0210 con 0x7f49200d3820 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f49200d04f0 con 0x7f4920162b40 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.219+0000 7f4924de4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f49200d07d0 con 0x7f49200d3c80 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491dd9b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f49200d3820 0x7f49200d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53560/0 (socket says 172.21.15.62:53560) 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491dd9b640 1 -- 172.21.15.62:0/1465386959 learned_addr learned my addr 172.21.15.62:0/1465386959 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491e59c640 1 --2- 172.21.15.62:0/1465386959 >> v2:172.21.15.174:3300/0 conn(0x7f4920162b40 0x7f49200d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491ed9d640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f49200d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2126509162 0 0) 0x7f49200d0210 con 0x7f49200d3820 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f490c003520 con 0x7f49200d3820 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 740832410 0 0) 0x7f490c003520 con 0x7f49200d3820 2024-04-26T22:28:21.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 msgr2=0x7f49200d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f49200d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 >> v2:172.21.15.174:3300/0 conn(0x7f4920162b40 msgr2=0x7f49200d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 --2- 172.21.15.62:0/1465386959 >> v2:172.21.15.174:3300/0 conn(0x7f4920162b40 0x7f49200d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200d0ab0 con 0x7f49200d3820 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f4908001670 con 0x7f49200d3820 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 >> v1:172.21.15.62:6789/0 conn(0x7f49200d3820 legacy=0x7f49200d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c0039b0 0x7f490c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c004800 0x7f490c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f490c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d04f0 con 0x7f490c004800 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d07d0 con 0x7f490c0039b0 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f490c00b9f0 con 0x7f49200d3c80 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491dd9b640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c004800 0x7f490c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491e59c640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c0039b0 0x7f490c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.227 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.220+0000 7f491ed9d640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f490c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 -- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 msgr2=0x7f490c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49200d3c80 0x7f490c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 -- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c0039b0 msgr2=0x7f490c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c0039b0 0x7f490c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 -- 172.21.15.62:0/1465386959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c003520 con 0x7f490c004800 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491dd9b640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c004800 0x7f490c004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4908001050 tx=0x7f490c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9083cae2b6ececc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.221+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490802e070 con 0x7f490c004800 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4908005af0 con 0x7f490c004800 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f491d59a640 1 -- 172.21.15.62:0/1465386959 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908007d70 con 0x7f490c004800 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 -- 172.21.15.62:0/1465386959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c004800 msgr2=0x7f490c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 --2- 172.21.15.62:0/1465386959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f490c004800 0x7f490c004c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4908001050 tx=0x7f490c005830 comp rx=0 tx=0).stop 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f491ed9d640 1 -- 172.21.15.62:0/1465386959 reap_dead start 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 -- 172.21.15.62:0/1465386959 shutdown_connections 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 -- 172.21.15.62:0/1465386959 >> 172.21.15.62:0/1465386959 conn(0x7f492009e550 msgr2=0x7f49200d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:21.228 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 -- 172.21.15.62:0/1465386959 shutdown_connections 2024-04-26T22:28:21.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.222+0000 7f4924de4640 1 -- 172.21.15.62:0/1465386959 wait complete. 2024-04-26T22:28:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 Processor -- start 2024-04-26T22:28:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 -- start start 2024-04-26T22:28:21.231 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c004800 0x7f49200ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49201f8920 0x7f49201f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49200b5bb0 con 0x7f49201f92e0 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49200d4110 con 0x7f490c004800 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f4924de4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f49200d6110 con 0x7f49201f8920 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f491ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f491ed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38752/0 (socket says 172.21.15.62:38752) 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.224+0000 7f491ed9d640 1 -- 172.21.15.62:0/1015170836 learned_addr learned my addr 172.21.15.62:0/1015170836 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491dd9b640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49201f8920 0x7f49201f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491e59c640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c004800 0x7f49200ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 -- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49201f8920 msgr2=0x7f49201f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49201f8920 0x7f49201f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 -- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c004800 msgr2=0x7f49200ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.232 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c004800 0x7f49200ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 -- 172.21.15.62:0/1015170836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200c9410 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f491ed9d640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f492006c710 tx=0x7f491000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491000fde0 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4910015430 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49200c96f0 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910015820 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.225+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49200c9cb0 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.226+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492006a360 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.227+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f49100159c0 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.227+0000 7f4916ffd640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 0x7f490006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.227+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f491008ca30 con 0x7f49201f92e0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.227+0000 7f491e59c640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 0x7f490006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.228+0000 7f491e59c640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 0x7f490006ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4918009a00 tx=0x7f4918008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.233 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.231+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4910019030 con 0x7f49201f92e0 2024-04-26T22:28:21.454 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.454+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49200d2960 con 0x7f49201f92e0 2024-04-26T22:28:21.455 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.455+0000 7f4916ffd640 1 -- 172.21.15.62:0/1015170836 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f4910010040 con 0x7f49201f92e0 2024-04-26T22:28:21.455 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:21.455 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:21.455 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 msgr2=0x7f490006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 0x7f490006ee40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4918009a00 tx=0x7f4918008040 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 msgr2=0x7f492015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f492006c710 tx=0x7f491000f580 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 shutdown_connections 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f490006c980 0x7f490006ee40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49201f92e0 0x7f492015f8f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49201f8920 0x7f49201f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 --2- 172.21.15.62:0/1015170836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c004800 0x7f49200ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.456+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 >> 172.21.15.62:0/1015170836 conn(0x7f492009e550 msgr2=0x7f49200d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.457+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 shutdown_connections 2024-04-26T22:28:21.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.457+0000 7f4924de4640 1 -- 172.21.15.62:0/1015170836 wait complete. 2024-04-26T22:28:21.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:21.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.570+0000 7fd143d38640 1 Processor -- start 2024-04-26T22:28:21.570 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.570+0000 7fd143d38640 1 -- start start 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd143d38640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd13c0d22e0 0x7fd13c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd143d38640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c15f760 0x7fd13c0b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd143d38640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13c0d3de0 con 0x7fd13c15f380 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd143d38640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c0d43a0 con 0x7fd13c0d22e0 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd143d38640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c0d40c0 con 0x7fd13c15f760 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd1412d3640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd13c0d22e0 0x7fd13c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd141ad4640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd13c15f380 0x7fd13c0d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53564/0 (socket says 172.21.15.62:53564) 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd141ad4640 1 -- 172.21.15.62:0/822610775 learned_addr learned my addr 172.21.15.62:0/822610775 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd1422d5640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c15f760 0x7fd13c0b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd1412d3640 1 -- 172.21.15.62:0/822610775 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c15f760 msgr2=0x7fd13c0b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd1412d3640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c15f760 0x7fd13c0b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.572 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.571+0000 7fd1412d3640 1 -- 172.21.15.62:0/822610775 >> v1:172.21.15.62:6789/0 conn(0x7fd13c15f380 legacy=0x7fd13c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.572+0000 7fd140ad2640 1 -- 172.21.15.62:0/822610775 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1026563863 0 0) 0x7fd13c0d3de0 con 0x7fd13c15f380 2024-04-26T22:28:21.573 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.572+0000 7fd1412d3640 1 -- 172.21.15.62:0/822610775 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c0b8280 con 0x7fd13c0d22e0 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.572+0000 7fd1412d3640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c0d22e0 0x7fd13c0b8a80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097a0 tx=0x7fd12c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211ec92ed3d99836 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.572+0000 7fd140ad2640 1 -- 172.21.15.62:0/822610775 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c037070 con 0x7fd13c0d22e0 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.572+0000 7fd140ad2640 1 -- 172.21.15.62:0/822610775 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c032440 con 0x7fd13c0d22e0 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd140ad2640 1 -- 172.21.15.62:0/822610775 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c032810 con 0x7fd13c0d22e0 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 -- 172.21.15.62:0/822610775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c0d22e0 msgr2=0x7fd13c0b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c0d22e0 0x7fd13c0b8a80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097a0 tx=0x7fd12c02a210 comp rx=0 tx=0).stop 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 -- 172.21.15.62:0/822610775 shutdown_connections 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c0d22e0 0x7fd13c0b8a80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 --2- 172.21.15.62:0/822610775 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c15f760 0x7fd13c0b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.574 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 -- 172.21.15.62:0/822610775 >> 172.21.15.62:0/822610775 conn(0x7fd13c09e560 msgr2=0x7fd13c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:21.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 -- 172.21.15.62:0/822610775 shutdown_connections 2024-04-26T22:28:21.575 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.573+0000 7fd143d38640 1 -- 172.21.15.62:0/822610775 wait complete. 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.574+0000 7fd143d38640 1 Processor -- start 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.574+0000 7fd143d38640 1 -- start start 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c0d22e0 0x7fd13c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c15f380 0x7fd13c1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0d1280 con 0x7fd13c15f760 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c0d1100 con 0x7fd13c15f380 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd143d38640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c0d5b50 con 0x7fd13c0d22e0 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1422d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1422d5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38770/0 (socket says 172.21.15.62:38770) 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1422d5640 1 -- 172.21.15.62:0/3003035413 learned_addr learned my addr 172.21.15.62:0/3003035413 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:21.577 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd141ad4640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c0d22e0 0x7fd13c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1412d3640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c15f380 0x7fd13c1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1422d5640 1 -- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c0d22e0 msgr2=0x7fd13c1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.575+0000 7fd1422d5640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c0d22e0 0x7fd13c1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1422d5640 1 -- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c15f380 msgr2=0x7fd13c1f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1422d5640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c15f380 0x7fd13c1f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1422d5640 1 -- 172.21.15.62:0/3003035413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c1f61c0 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1422d5640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd13800efc0 tx=0x7fd13800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138018020 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd138014480 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c1595b0 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138014870 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.576+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c161880 con 0x7fd13c15f760 2024-04-26T22:28:21.578 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.577+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c06a360 con 0x7fd13c15f760 2024-04-26T22:28:21.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.577+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd138005d10 con 0x7fd13c15f760 2024-04-26T22:28:21.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.578+0000 7fd1327fc640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 0x7fd12006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.578+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd13808c5b0 con 0x7fd13c15f760 2024-04-26T22:28:21.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.578+0000 7fd141ad4640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 0x7fd12006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.579+0000 7fd141ad4640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 0x7fd12006ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f60 tx=0x7fd134007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.582+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd138022070 con 0x7fd13c15f760 2024-04-26T22:28:21.769 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.769+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd13c1eb110 con 0x7fd13c15f760 2024-04-26T22:28:21.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.770+0000 7fd1327fc640 1 -- 172.21.15.62:0/3003035413 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7fd13c1f61c0 con 0x7fd13c15f760 2024-04-26T22:28:21.770 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:21.771 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 msgr2=0x7fd12006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 0x7fd12006ee40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd134000f60 tx=0x7fd134007d50 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 msgr2=0x7fd13c1f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd13800efc0 tx=0x7fd13800c490 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 shutdown_connections 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd12006c980 0x7fd12006ee40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd13c15f760 0x7fd13c1f7f60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c15f380 0x7fd13c1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 --2- 172.21.15.62:0/3003035413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd13c0d22e0 0x7fd13c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 >> 172.21.15.62:0/3003035413 conn(0x7fd13c09e560 msgr2=0x7fd13c0b8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 shutdown_connections 2024-04-26T22:28:21.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.772+0000 7fd143d38640 1 -- 172.21.15.62:0/3003035413 wait complete. 2024-04-26T22:28:21.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T22:28:00.329554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T22:28:00.858734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:28:21.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:28:21.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.889+0000 7f77082a2640 1 Processor -- start 2024-04-26T22:28:21.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.889+0000 7f77082a2640 1 -- start start 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f77082a2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f770015f400 0x7f77000b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f77082a2640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000b2760 con 0x7f77000b4cf0 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f77082a2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77000b2a00 con 0x7f77000d5a80 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f77082a2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f77000b43e0 con 0x7f770015f400 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f770683f640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f77000b4cf0 0x7f77001630b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53574/0 (socket says 172.21.15.62:53574) 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f770683f640 1 -- 172.21.15.62:0/4220304366 learned_addr learned my addr 172.21.15.62:0/4220304366 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:21.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.890+0000 7f770603e640 1 --2- 172.21.15.62:0/4220304366 >> v2:172.21.15.174:3300/0 conn(0x7f770015f400 0x7f77000b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906913881 0 0) 0x7f77000b2760 con 0x7f77000b4cf0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e8003520 con 0x7f77000b4cf0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605770724 0 0) 0x7f77000b2a00 con 0x7f77000d5a80 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77000b2760 con 0x7f77000d5a80 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2727272193 0 0) 0x7f76e8003520 con 0x7f77000b4cf0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 >> v2:172.21.15.174:3300/0 conn(0x7f770015f400 msgr2=0x7f77000b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 --2- 172.21.15.62:0/4220304366 >> v2:172.21.15.174:3300/0 conn(0x7f770015f400 0x7f77000b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 >> v1:172.21.15.174:6789/0 conn(0x7f77000d5a80 legacy=0x7f77000b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77000b2ce0 con 0x7f77000b4cf0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f76f0001a10 con 0x7f77000b4cf0 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 >> v1:172.21.15.62:6789/0 conn(0x7f77000b4cf0 legacy=0x7f77001630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f76e8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76e8005210 0x7f76e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f76e800c110 con 0x7f76e8003b90 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76e800c2c0 con 0x7f76e8005210 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f76e800c4c0 con 0x7f76e8004800 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f76e8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770583d640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770683f640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76e8005210 0x7f76e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 -- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76e8004800 msgr2=0x7f76e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76e8004800 0x7f76e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 -- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76e8005210 msgr2=0x7f76e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76e8005210 0x7f76e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770603e640 1 -- 172.21.15.62:0/4220304366 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e8003520 con 0x7f76e8003b90 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.891+0000 7f770683f640 1 -- 172.21.15.62:0/4220304366 reap_dead start 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.892+0000 7f770603e640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f76e8003f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009ac0 tx=0x7f76e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6adbb40f52fab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.892+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8036070 con 0x7f76e8003b90 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.892+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f8031400 con 0x7f76e8003b90 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.892+0000 7f770503c640 1 -- 172.21.15.62:0/4220304366 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8009690 con 0x7f76e8003b90 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 -- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 msgr2=0x7f76e8003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f76e8003f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009ac0 tx=0x7f76e8005830 comp rx=0 tx=0).stop 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 -- 172.21.15.62:0/4220304366 shutdown_connections 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 --2- 172.21.15.62:0/4220304366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f76e8003f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 -- 172.21.15.62:0/4220304366 >> 172.21.15.62:0/4220304366 conn(0x7f770009e4f0 msgr2=0x7f77000d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 -- 172.21.15.62:0/4220304366 shutdown_connections 2024-04-26T22:28:21.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.893+0000 7f77082a2640 1 -- 172.21.15.62:0/4220304366 wait complete. 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.894+0000 7f77082a2640 1 Processor -- start 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.894+0000 7f77082a2640 1 -- start start 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 0x7f7700162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77000cadb0 0x7f77000cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f770015f7e0 con 0x7f76e8003b90 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77000b32a0 con 0x7f7700159430 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f770015f960 con 0x7f77000cadb0 2024-04-26T22:28:21.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38788/0 (socket says 172.21.15.62:38788) 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 -- 172.21.15.62:0/403067338 learned_addr learned my addr 172.21.15.62:0/403067338 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770583d640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 0x7f7700162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770683f640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77000cadb0 0x7f77000cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 -- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77000cadb0 msgr2=0x7f77000cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77000cadb0 0x7f77000cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 -- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 msgr2=0x7f7700162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 0x7f7700162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 -- 172.21.15.62:0/403067338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77000c9510 con 0x7f76e8003b90 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770583d640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 0x7f7700162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f770603e640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76f8034040 tx=0x7f76f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f803f020 con 0x7f76e8003b90 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f8037040 con 0x7f76e8003b90 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77000c97b0 con 0x7f76e8003b90 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.895+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f8036070 con 0x7f76e8003b90 2024-04-26T22:28:21.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.896+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77000c9d70 con 0x7f76e8003b90 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.897+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770006a360 con 0x7f76e8003b90 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.897+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f76f8042020 con 0x7f76e8003b90 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.897+0000 7f76feffd640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 0x7f76ec06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.898+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f76f80abcc0 con 0x7f76e8003b90 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.898+0000 7f770583d640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 0x7f76ec06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:21.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.898+0000 7f770583d640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 0x7f76ec06ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007ae0 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:21.902 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:21.902+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76f8049030 con 0x7f76e8003b90 2024-04-26T22:28:22.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.086+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77000cffe0 con 0x7f76e8003b90 2024-04-26T22:28:22.086 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.086+0000 7f76feffd640 1 -- 172.21.15.62:0/403067338 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) ==== 93+0+10 (secure 0 0 0) 0x7f76f807a1c0 con 0x7f76e8003b90 2024-04-26T22:28:22.086 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 msgr2=0x7f76ec06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 0x7f76ec06ee40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f76f4007ae0 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 msgr2=0x7f7700159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76f8034040 tx=0x7f76f8004730 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 shutdown_connections 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f76ec06c980 0x7f76ec06ee40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f77000cadb0 0x7f77000cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7700159430 0x7f7700162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 --2- 172.21.15.62:0/403067338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f76e8003b90 0x7f7700159980 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 >> 172.21.15.62:0/403067338 conn(0x7f770009e4f0 msgr2=0x7f77000b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 shutdown_connections 2024-04-26T22:28:22.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.088+0000 7f77082a2640 1 -- 172.21.15.62:0/403067338 wait complete. 2024-04-26T22:28:22.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T22:28:22.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.203+0000 7f02b787b640 1 Processor -- start 2024-04-26T22:28:22.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.203+0000 7f02b787b640 1 -- start start 2024-04-26T22:28:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b787b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 0x7f02b00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b787b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b787b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b00b2550 con 0x7f02b015f9c0 2024-04-26T22:28:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b787b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f02b00d0340 con 0x7f02b00cf990 2024-04-26T22:28:22.204 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b787b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f02b00d1f70 con 0x7f02b00b2750 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b5e18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b4e16640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 0x7f02b00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b5e18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:48640/0 (socket says 172.21.15.62:48640) 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b4e16640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 0x7f02b00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:52786/0 (socket says 172.21.15.62:52786) 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b5617640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f02b015f9c0 0x7f02b00d1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:43338/0 (socket says 172.21.15.62:43338) 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.204+0000 7f02b5e18640 1 -- 172.21.15.62:0/2261710224 learned_addr learned my addr 172.21.15.62:0/2261710224 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:22.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.205+0000 7f02b5e18640 1 -- 172.21.15.62:0/2261710224 >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 msgr2=0x7f02b00cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:22.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.205+0000 7f02b5e18640 1 --2- 172.21.15.62:0/2261710224 >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 0x7f02b00cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.205+0000 7f02b5e18640 1 -- 172.21.15.62:0/2261710224 >> v1:172.21.15.174:6789/0 conn(0x7f02b015f9c0 legacy=0x7f02b00d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.205+0000 7f02b5e18640 1 -- 172.21.15.62:0/2261710224 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b006dcf0 con 0x7f02b00b2750 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02b5e18640 1 --2- 172.21.15.62:0/2261710224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009870 tx=0x7f02a802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c123f2b539be04c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02a7fff640 1 -- 172.21.15.62:0/2261710224 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8037070 con 0x7f02b00b2750 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02a7fff640 1 -- 172.21.15.62:0/2261710224 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a8032440 con 0x7f02b00b2750 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02a7fff640 1 -- 172.21.15.62:0/2261710224 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8032810 con 0x7f02b00b2750 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02b787b640 1 -- 172.21.15.62:0/2261710224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 msgr2=0x7f02b00b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.206+0000 7f02b787b640 1 --2- 172.21.15.62:0/2261710224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009870 tx=0x7f02a802a1c0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 -- 172.21.15.62:0/2261710224 shutdown_connections 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 --2- 172.21.15.62:0/2261710224 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b00b2be0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.207 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 --2- 172.21.15.62:0/2261710224 >> v2:172.21.15.174:3300/0 conn(0x7f02b00cf990 0x7f02b00cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 -- 172.21.15.62:0/2261710224 >> 172.21.15.62:0/2261710224 conn(0x7f02b00d2f80 msgr2=0x7f02b00d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 -- 172.21.15.62:0/2261710224 shutdown_connections 2024-04-26T22:28:22.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.207+0000 7f02b787b640 1 -- 172.21.15.62:0/2261710224 wait complete. 2024-04-26T22:28:22.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.208+0000 7f02b787b640 1 Processor -- start 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.208+0000 7f02b787b640 1 -- start start 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.208+0000 7f02b787b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b015f9c0 0x7f02b0150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f02b006e590 con 0x7f02b00cf990 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02b006e2b0 con 0x7f02b015f9c0 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f02b006e870 con 0x7f02b00b2750 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38790/0 (socket says 172.21.15.62:38790) 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 -- 172.21.15.62:0/2160605052 learned_addr learned my addr 172.21.15.62:0/2160605052 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b5617640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b5e18640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b015f9c0 0x7f02b0150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.210 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 -- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 msgr2=0x7f02b0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 -- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b015f9c0 msgr2=0x7f02b0150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b015f9c0 0x7f02b0150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 -- 172.21.15.62:0/2160605052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b01eb5e0 con 0x7f02b00cf990 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b4e16640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9b0 tx=0x7f02a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0017070 con 0x7f02b00cf990 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a000c9a0 con 0x7f02b00cf990 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b01ea540 con 0x7f02b00cf990 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.209+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a000cd90 con 0x7f02b00cf990 2024-04-26T22:28:22.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.210+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01eab00 con 0x7f02b00cf990 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.210+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b006a360 con 0x7f02b00cf990 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.211+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f02a001a070 con 0x7f02b00cf990 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.211+0000 7f02a5ffb640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 0x7f029406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.211+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f02a008bad0 con 0x7f02b00cf990 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.212+0000 7f02b5617640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 0x7f029406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.212+0000 7f02b5617640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 0x7f029406ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02ac007fd0 tx=0x7f02ac001630 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.215 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.216+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f02a00598c0 con 0x7f02b00cf990 2024-04-26T22:28:22.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.401+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f02b00b1840 con 0x7f02b00cf990 2024-04-26T22:28:22.402 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.402+0000 7f02a5ffb640 1 -- 172.21.15.62:0/2160605052 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v31) ==== 104+0+11 (secure 0 0 0) 0x7f02a005dbc0 con 0x7f02b00cf990 2024-04-26T22:28:22.402 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 32 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 msgr2=0x7f029406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 0x7f029406ee40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f02ac007fd0 tx=0x7f02ac001630 comp rx=0 tx=0).stop 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 msgr2=0x7f02b0155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f02a000e9b0 tx=0x7f02a000ee70 comp rx=0 tx=0).stop 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 shutdown_connections 2024-04-26T22:28:22.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f029406c980 0x7f029406ee40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02b015f9c0 0x7f02b0150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f02b00cf990 0x7f02b0155ee0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 --2- 172.21.15.62:0/2160605052 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f02b00b2750 0x7f02b0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 >> 172.21.15.62:0/2160605052 conn(0x7f02b00d2f80 msgr2=0x7f02b015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 shutdown_connections 2024-04-26T22:28:22.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.404+0000 7f02b787b640 1 -- 172.21.15.62:0/2160605052 wait complete. 2024-04-26T22:28:22.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T22:28:22.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.519+0000 7ff1c117f640 1 Processor -- start 2024-04-26T22:28:22.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.519+0000 7ff1c117f640 1 -- start start 2024-04-26T22:28:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.519+0000 7ff1c117f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 0x7ff1bc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1c117f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc0cd640 con 0x7ff1bc0d26f0 2024-04-26T22:28:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1c117f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc0cd920 con 0x7ff1bc0cf870 2024-04-26T22:28:22.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1c117f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc0cdc00 con 0x7ff1bc0b2290 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bb59e640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7ff1bc0d26f0 0x7ff1bc15ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:53578/0 (socket says 172.21.15.62:53578) 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bb59e640 1 -- 172.21.15.62:0/992037973 learned_addr learned my addr 172.21.15.62:0/992037973 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bad9d640 1 --2- 172.21.15.62:0/992037973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 0x7ff1bc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2578010934 0 0) 0x7ff1bc0cd640 con 0x7ff1bc0d26f0 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1a4003520 con 0x7ff1bc0d26f0 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174766553 0 0) 0x7ff1bc0cd920 con 0x7ff1bc0cf870 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1bc0cd640 con 0x7ff1bc0cf870 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bad9d640 1 -- 172.21.15.62:0/992037973 >> v1:172.21.15.174:6789/0 conn(0x7ff1bc0cf870 legacy=0x7ff1bc0d1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bad9d640 1 -- 172.21.15.62:0/992037973 >> v1:172.21.15.62:6789/0 conn(0x7ff1bc0d26f0 legacy=0x7ff1bc15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 126605667 0 0) 0x7ff1a4003520 con 0x7ff1bc0d26f0 2024-04-26T22:28:22.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.520+0000 7ff1bad9d640 1 -- 172.21.15.62:0/992037973 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc0cdee0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.521+0000 7ff1bad9d640 1 --2- 172.21.15.62:0/992037973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 0x7ff1bc0b2930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0097d0 tx=0x7ff1ac02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc8c0990298c509 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.521+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac037070 con 0x7ff1bc0b2290 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.521+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac02ac30 con 0x7ff1bc0b2290 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.521+0000 7ff1b9d9b640 1 -- 172.21.15.62:0/992037973 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40019f0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 -- 172.21.15.62:0/992037973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 msgr2=0x7ff1bc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 --2- 172.21.15.62:0/992037973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 0x7ff1bc0b2930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0097d0 tx=0x7ff1ac02a1a0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 -- 172.21.15.62:0/992037973 shutdown_connections 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 --2- 172.21.15.62:0/992037973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0b2290 0x7ff1bc0b2930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 -- 172.21.15.62:0/992037973 >> 172.21.15.62:0/992037973 conn(0x7ff1bc0d2ff0 msgr2=0x7ff1bc0d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 -- 172.21.15.62:0/992037973 shutdown_connections 2024-04-26T22:28:22.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.522+0000 7ff1c117f640 1 -- 172.21.15.62:0/992037973 wait complete. 2024-04-26T22:28:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.523+0000 7ff1c117f640 1 Processor -- start 2024-04-26T22:28:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.523+0000 7ff1c117f640 1 -- start start 2024-04-26T22:28:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1bc0cf870 0x7ff1bc0c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0d26f0 0x7ff1bc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc0cfcd0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc0b2670 con 0x7ff1bc0cf870 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1c117f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc0cfe50 con 0x7ff1bc0d26f0 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38800/0 (socket says 172.21.15.62:38800) 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 -- 172.21.15.62:0/1422320206 learned_addr learned my addr 172.21.15.62:0/1422320206 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bb59e640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0d26f0 0x7ff1bc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1ba59c640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1bc0cf870 0x7ff1bc0c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 -- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0d26f0 msgr2=0x7ff1bc0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0d26f0 0x7ff1bc0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 -- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1bc0cf870 msgr2=0x7ff1bc0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1bc0cf870 0x7ff1bc0c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 -- 172.21.15.62:0/1422320206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc0c83d0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.524+0000 7ff1bad9d640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac035040 tx=0x7ff1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.525+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac041020 con 0x7ff1bc0b2290 2024-04-26T22:28:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.525+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac03d640 con 0x7ff1bc0b2290 2024-04-26T22:28:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.525+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc0c7330 con 0x7ff1bc0b2290 2024-04-26T22:28:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.525+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac037040 con 0x7ff1bc0b2290 2024-04-26T22:28:22.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.525+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc0c78f0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.526+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc06a360 con 0x7ff1bc0b2290 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.526+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff1ac044020 con 0x7ff1bc0b2290 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.527+0000 7ff1a37fe640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 0x7ff1a806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.527+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7ff1ac0ac0a0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.527+0000 7ff1ba59c640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 0x7ff1a806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.527+0000 7ff1ba59c640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 0x7ff1a806ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc153d30 tx=0x7ff1b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.531+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff1ac04b030 con 0x7ff1bc0b2290 2024-04-26T22:28:22.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.716+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1bc0744b0 con 0x7ff1bc0b2290 2024-04-26T22:28:22.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.717+0000 7ff1a37fe640 1 -- 172.21.15.62:0/1422320206 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v31) ==== 100+0+11 (secure 0 0 0) 0x7ff1ac07a570 con 0x7ff1bc0b2290 2024-04-26T22:28:22.717 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 64 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 msgr2=0x7ff1a806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 0x7ff1a806ee40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc153d30 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 msgr2=0x7ff1bc153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac035040 tx=0x7ff1ac0046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 shutdown_connections 2024-04-26T22:28:22.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff1a806c980 0x7ff1a806ee40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff1bc0d26f0 0x7ff1bc0cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1bc0cf870 0x7ff1bc0c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 --2- 172.21.15.62:0/1422320206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1bc0b2290 0x7ff1bc153620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 >> 172.21.15.62:0/1422320206 conn(0x7ff1bc0d2ff0 msgr2=0x7ff1bc0b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 shutdown_connections 2024-04-26T22:28:22.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.719+0000 7ff1c117f640 1 -- 172.21.15.62:0/1422320206 wait complete. 2024-04-26T22:28:22.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:22.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.830+0000 7f0dc7ac8640 1 Processor -- start 2024-04-26T22:28:22.831 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 -- start start 2024-04-26T22:28:22.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0dc00d3820 0x7f0dc00d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc00d0210 con 0x7f0dc0162b40 2024-04-26T22:28:22.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc00d41c0 con 0x7f0dc00d3820 2024-04-26T22:28:22.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.831+0000 7f0dc7ac8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00d04f0 con 0x7f0dc00b4ba0 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0dc00d3820 0x7f0dc00d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f0dc00d3820 0x7f0dc00d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36178/0 (socket says 172.21.15.62:36178) 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 -- 172.21.15.62:0/185662507 learned_addr learned my addr 172.21.15.62:0/185662507 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc6065640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 -- 172.21.15.62:0/185662507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 msgr2=0x7f0dc00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 -- 172.21.15.62:0/185662507 >> v1:172.21.15.174:6789/0 conn(0x7f0dc0162b40 legacy=0x7f0dc00d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.832+0000 7f0dc5063640 1 -- 172.21.15.62:0/185662507 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00d07d0 con 0x7f0dc00d3820 2024-04-26T22:28:22.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc5063640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc00d3c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009870 tx=0x7f0db402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90933df0a34458dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc4862640 1 -- 172.21.15.62:0/185662507 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4037070 con 0x7f0dc00d3820 2024-04-26T22:28:22.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc4862640 1 -- 172.21.15.62:0/185662507 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db4032440 con 0x7f0dc00d3820 2024-04-26T22:28:22.834 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc4862640 1 -- 172.21.15.62:0/185662507 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4032810 con 0x7f0dc00d3820 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/185662507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 msgr2=0x7f0dc00d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc00d3c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009870 tx=0x7f0db402a270 comp rx=0 tx=0).stop 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.833+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/185662507 shutdown_connections 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.834+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.834+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/185662507 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc00d3c80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.834+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/185662507 >> 172.21.15.62:0/185662507 conn(0x7f0dc009e550 msgr2=0x7f0dc00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.834+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/185662507 shutdown_connections 2024-04-26T22:28:22.835 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.834+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/185662507 wait complete. 2024-04-26T22:28:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.835+0000 7f0dc7ac8640 1 Processor -- start 2024-04-26T22:28:22.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.835+0000 7f0dc7ac8640 1 -- start start 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc0161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 0x7f0dc0155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00d12b0 con 0x7f0dc00d3820 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00d0d90 con 0x7f0dc0162b40 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00cfe30 con 0x7f0dc00b4ba0 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36194/0 (socket says 172.21.15.62:36194) 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 -- 172.21.15.62:0/3432545225 learned_addr learned my addr 172.21.15.62:0/3432545225 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc6065640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 0x7f0dc0155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5864640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc0161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 -- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 msgr2=0x7f0dc0161210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc0161210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 -- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 msgr2=0x7f0dc0155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 0x7f0dc0155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 -- 172.21.15.62:0/3432545225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00b8b50 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5864640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc0161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc6065640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 0x7f0dc0155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc5063640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4041020 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db40306c0 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc00b8e30 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.836+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4037040 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.837+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc00b93f0 con 0x7f0dc00d3820 2024-04-26T22:28:22.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.837+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc006a360 con 0x7f0dc00d3820 2024-04-26T22:28:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.838+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0db4044020 con 0x7f0dc00d3820 2024-04-26T22:28:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.838+0000 7f0db27fc640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 0x7f0da406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.839+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f0db40ac340 con 0x7f0dc00d3820 2024-04-26T22:28:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.839+0000 7f0dc5864640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 0x7f0da406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:22.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.839+0000 7f0dc5864640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 0x7f0da406ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc001700 tx=0x7f0dbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:22.842 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:22.843+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0db404b030 con 0x7f0dc00d3820 2024-04-26T22:28:23.067 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.067+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dc00d2960 con 0x7f0dc00d3820 2024-04-26T22:28:23.068 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.068+0000 7f0db27fc640 1 -- 172.21.15.62:0/3432545225 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7f0db407a810 con 0x7f0dc00d3820 2024-04-26T22:28:23.068 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:23.068 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:23.069 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-26T22:28:23.070 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 msgr2=0x7f0da406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 0x7f0da406ee40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc001700 tx=0x7f0dbc009040 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 msgr2=0x7f0dc01552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db40046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 shutdown_connections 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0da406c980 0x7f0da406ee40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0dc0162b40 0x7f0dc0155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0dc00d3820 0x7f0dc01552d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 --2- 172.21.15.62:0/3432545225 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0dc00b4ba0 0x7f0dc0161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 >> 172.21.15.62:0/3432545225 conn(0x7f0dc009e550 msgr2=0x7f0dc006e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.071 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 shutdown_connections 2024-04-26T22:28:23.072 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.070+0000 7f0dc7ac8640 1 -- 172.21.15.62:0/3432545225 wait complete. 2024-04-26T22:28:23.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.182+0000 7fce654f1640 1 Processor -- start 2024-04-26T22:28:23.182 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.182+0000 7fce654f1640 1 -- start start 2024-04-26T22:28:23.186 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce654f1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fce601600c0 0x7fce600cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce654f1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce60160530 con 0x7fce6015fc10 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce654f1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce60072e30 con 0x7fce60159a90 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce654f1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fce60073110 con 0x7fce601600c0 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce57fff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fce6015fc10 0x7fce600ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57688/0 (socket says 172.21.15.62:57688) 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce57fff640 1 -- 172.21.15.62:0/225571181 learned_addr learned my addr 172.21.15.62:0/225571181 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce5f7fe640 1 --2- 172.21.15.62:0/225571181 >> v2:172.21.15.174:3300/0 conn(0x7fce601600c0 0x7fce600cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438748905 0 0) 0x7fce60160530 con 0x7fce6015fc10 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce4c003520 con 0x7fce6015fc10 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 105733123 0 0) 0x7fce60072e30 con 0x7fce60159a90 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.183+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce60160530 con 0x7fce60159a90 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 590370960 0 0) 0x7fce4c003520 con 0x7fce6015fc10 2024-04-26T22:28:23.187 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 >> v2:172.21.15.174:3300/0 conn(0x7fce601600c0 msgr2=0x7fce600cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 --2- 172.21.15.62:0/225571181 >> v2:172.21.15.174:3300/0 conn(0x7fce601600c0 0x7fce600cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 >> v1:172.21.15.174:6789/0 conn(0x7fce60159a90 legacy=0x7fce6006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce600733f0 con 0x7fce6015fc10 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fce480015f0 con 0x7fce6015fc10 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 >> v1:172.21.15.62:6789/0 conn(0x7fce6015fc10 legacy=0x7fce600ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce4c003b90 0x7fce4c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce4c004800 0x7fce4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce60073110 con 0x7fce4c005280 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c00c1c0 con 0x7fce4c004800 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c00c3c0 con 0x7fce4c003b90 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce57fff640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce4c004800 0x7fce4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.188 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5effd640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce4c003b90 0x7fce4c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 -- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce4c003b90 msgr2=0x7fce4c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce4c003b90 0x7fce4c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 -- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce4c004800 msgr2=0x7fce4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce4c004800 0x7fce4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce5f7fe640 1 -- 172.21.15.62:0/225571181 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c003520 con 0x7fce4c005280 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.184+0000 7fce57fff640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce4c004800 0x7fce4c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce5f7fe640 1 -- 172.21.15.62:0/225571181 reap_dead start 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce5f7fe640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce4c00b9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fce58009fc0 tx=0x7fce4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3b32ec282c2ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58009c10 con 0x7fce4c005280 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce58003b90 con 0x7fce4c005280 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce5e7fc640 1 -- 172.21.15.62:0/225571181 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58004020 con 0x7fce4c005280 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce654f1640 1 -- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 msgr2=0x7fce4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce654f1640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce4c00b9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fce58009fc0 tx=0x7fce4c005830 comp rx=0 tx=0).stop 2024-04-26T22:28:23.189 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce654f1640 1 -- 172.21.15.62:0/225571181 shutdown_connections 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce654f1640 1 --2- 172.21.15.62:0/225571181 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce4c00b9c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.185+0000 7fce654f1640 1 -- 172.21.15.62:0/225571181 >> 172.21.15.62:0/225571181 conn(0x7fce600d0d50 msgr2=0x7fce600d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.186+0000 7fce654f1640 1 -- 172.21.15.62:0/225571181 shutdown_connections 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.186+0000 7fce654f1640 1 -- 172.21.15.62:0/225571181 wait complete. 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 Processor -- start 2024-04-26T22:28:23.190 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 -- start start 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce60149940 0x7fce6014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce6014e150 0x7fce6014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fce60159e70 con 0x7fce4c005280 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fce601609d0 con 0x7fce60149940 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce654f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fce60159ff0 con 0x7fce6014e150 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce5effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce5effd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36198/0 (socket says 172.21.15.62:36198) 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.187+0000 7fce5effd640 1 -- 172.21.15.62:0/304542898 learned_addr learned my addr 172.21.15.62:0/304542898 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce57fff640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce60149940 0x7fce6014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 -- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce6014e150 msgr2=0x7fce6014e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.193 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce6014e150 0x7fce6014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 -- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce60149940 msgr2=0x7fce6014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce60149940 0x7fce6014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 -- 172.21.15.62:0/304542898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6014c850 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce5effd640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e580 tx=0x7fce5000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500052e0 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50018070 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6014cb30 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50003830 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.188+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6014d0f0 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.189+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6006a360 con 0x7fce4c005280 2024-04-26T22:28:23.194 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.190+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fce500039d0 con 0x7fce4c005280 2024-04-26T22:28:23.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.190+0000 7fce577fe640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 0x7fce4006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.190+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fce5008b900 con 0x7fce4c005280 2024-04-26T22:28:23.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.191+0000 7fce57fff640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 0x7fce4006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.191+0000 7fce57fff640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 0x7fce4006ee70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce48001550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.195 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.195+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce50059d50 con 0x7fce4c005280 2024-04-26T22:28:23.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.386+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce600d0730 con 0x7fce4c005280 2024-04-26T22:28:23.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.387+0000 7fce577fe640 1 -- 172.21.15.62:0/304542898 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7fce50059650 con 0x7fce4c005280 2024-04-26T22:28:23.387 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:23.387 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:23.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 msgr2=0x7fce4006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 0x7fce4006ee70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce48001550 comp rx=0 tx=0).stop 2024-04-26T22:28:23.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 msgr2=0x7fce60151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e580 tx=0x7fce5000ea40 comp rx=0 tx=0).stop 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 shutdown_connections 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fce4006c9b0 0x7fce4006ee70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fce6014e150 0x7fce6014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce60149940 0x7fce6014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 --2- 172.21.15.62:0/304542898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fce4c005280 0x7fce60151b70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 >> 172.21.15.62:0/304542898 conn(0x7fce600d0d50 msgr2=0x7fce6006f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 shutdown_connections 2024-04-26T22:28:23.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.389+0000 7fce654f1640 1 -- 172.21.15.62:0/304542898 wait complete. 2024-04-26T22:28:23.400 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T22:28:23.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:23.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.501+0000 7fd74f389640 1 Processor -- start 2024-04-26T22:28:23.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.501+0000 7fd74f389640 1 -- start start 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74f389640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd7480d22e0 0x7fd7480b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74f389640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7480b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74f389640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7480d3de0 con 0x7fd74815f380 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74f389640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd7480d43a0 con 0x7fd7480d22e0 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74f389640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd7480d40c0 con 0x7fd74815f760 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74d125640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd74815f380 0x7fd7480d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57696/0 (socket says 172.21.15.62:57696) 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74d125640 1 -- 172.21.15.62:0/3893369443 learned_addr learned my addr 172.21.15.62:0/3893369443 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:23.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> v2:172.21.15.174:3300/0 conn(0x7fd7480d22e0 0x7fd7480b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd74d926640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7480b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33141044 0 0) 0x7fd7480d3de0 con 0x7fd74815f380 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.502+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd730003520 con 0x7fd74815f380 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879029488 0 0) 0x7fd730003520 con 0x7fd74815f380 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 msgr2=0x7fd7480b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7480b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 >> v2:172.21.15.174:3300/0 conn(0x7fd7480d22e0 msgr2=0x7fd7480b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 --2- 172.21.15.62:0/3893369443 >> v2:172.21.15.174:3300/0 conn(0x7fd7480d22e0 0x7fd7480b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7480b8280 con 0x7fd74815f380 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fd740001670 con 0x7fd74815f380 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> v2:172.21.15.174:3300/0 conn(0x7fd7480d22e0 0x7fd7480b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 >> v1:172.21.15.62:6789/0 conn(0x7fd74815f380 legacy=0x7fd7480d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.506 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd730004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd730004d40 0x7fd7300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480d43a0 con 0x7fd730003f30 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480d40c0 con 0x7fd730004d40 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd73000b9f0 con 0x7fd74815f760 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd730004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74d125640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74d926640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd730004d40 0x7fd7300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 -- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 msgr2=0x7fd7300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74815f760 0x7fd7300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 -- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd730004d40 msgr2=0x7fd7300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd730004d40 0x7fd7300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.503+0000 7fd74c924640 1 -- 172.21.15.62:0/3893369443 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd730003520 con 0x7fd730003f30 2024-04-26T22:28:23.507 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd74c924640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd730004800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009c90 tx=0x7fd730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e86d1a1906a159 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c036070 con 0x7fd730003f30 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c031400 con 0x7fd730003f30 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd747fff640 1 -- 172.21.15.62:0/3893369443 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c009860 con 0x7fd730003f30 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd74f389640 1 -- 172.21.15.62:0/3893369443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 msgr2=0x7fd730004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd74f389640 1 --2- 172.21.15.62:0/3893369443 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd730004800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009c90 tx=0x7fd730005830 comp rx=0 tx=0).stop 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.504+0000 7fd74d926640 1 -- 172.21.15.62:0/3893369443 reap_dead start 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.505+0000 7fd74f389640 1 -- 172.21.15.62:0/3893369443 shutdown_connections 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.505+0000 7fd74f389640 1 -- 172.21.15.62:0/3893369443 >> 172.21.15.62:0/3893369443 conn(0x7fd74809e560 msgr2=0x7fd7480d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.505+0000 7fd74f389640 1 -- 172.21.15.62:0/3893369443 shutdown_connections 2024-04-26T22:28:23.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.505+0000 7fd74f389640 1 -- 172.21.15.62:0/3893369443 wait complete. 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.506+0000 7fd74f389640 1 Processor -- start 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.506+0000 7fd74f389640 1 -- start start 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.506+0000 7fd74f389640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.506+0000 7fd74f389640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7481f8e70 0x7fd7480b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74f389640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7481f92f0 0x7fd7480b7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74f389640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480d1280 con 0x7fd730003f30 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74f389640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7480d1100 con 0x7fd7481f8e70 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74f389640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd7480d5b50 con 0x7fd7481f92f0 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36222/0 (socket says 172.21.15.62:36222) 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 -- 172.21.15.62:0/719510504 learned_addr learned my addr 172.21.15.62:0/719510504 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 -- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7481f92f0 msgr2=0x7fd7480b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7481f92f0 0x7fd7480b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.511 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 -- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7481f8e70 msgr2=0x7fd7480b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7481f8e70 0x7fd7480b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 -- 172.21.15.62:0/719510504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd748150600 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74d125640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7400072f0 tx=0x7fd7400077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74001a070 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74000f9c0 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74000fdb0 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.507+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481508e0 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.508+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd748150ea0 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.508+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74806a360 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.509+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd74001d360 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.509+0000 7fd745ffb640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 0x7fd73406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.509+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd74008b600 con 0x7fd730003f30 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.509+0000 7fd74c924640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 0x7fd73406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.510+0000 7fd74c924640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 0x7fd73406ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd73c033000 tx=0x7fd73c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.513+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd740059ad0 con 0x7fd730003f30 2024-04-26T22:28:23.739 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.739+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7480d2d90 con 0x7fd730003f30 2024-04-26T22:28:23.740 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.739+0000 7fd745ffb640 1 -- 172.21.15.62:0/719510504 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7fd7400593d0 con 0x7fd730003f30 2024-04-26T22:28:23.740 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:23.740 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:23.740 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 msgr2=0x7fd73406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 0x7fd73406ee40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd73c033000 tx=0x7fd73c02e000 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 msgr2=0x7fd7480b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd7400072f0 tx=0x7fd7400077b0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 shutdown_connections 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd73406c980 0x7fd73406ee40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7481f92f0 0x7fd7480b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7481f8e70 0x7fd7480b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 --2- 172.21.15.62:0/719510504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd730003f30 0x7fd7480b5830 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.742 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.741+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 >> 172.21.15.62:0/719510504 conn(0x7fd74809e560 msgr2=0x7fd748161c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.742+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 shutdown_connections 2024-04-26T22:28:23.743 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.742+0000 7fd74f389640 1 -- 172.21.15.62:0/719510504 wait complete. 2024-04-26T22:28:23.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:23.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 Processor -- start 2024-04-26T22:28:23.854 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 -- start start 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd89c15f400 0x7fd89c0d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd89c0d0b80 0x7fd89c0d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c0d3220 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd89c0d26c0 con 0x7fd89c0d0b80 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.854+0000 7fd8a19c1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd89c0d5ef0 con 0x7fd89c15f400 2024-04-26T22:28:23.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd89c0d0b80 0x7fd89c0d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd89b7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd89c0d0ff0 0x7fd89c0b1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57710/0 (socket says 172.21.15.62:57710) 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd89b7fe640 1 -- 172.21.15.62:0/1219603029 learned_addr learned my addr 172.21.15.62:0/1219603029 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 --2- 172.21.15.62:0/1219603029 >> v2:172.21.15.62:3300/0 conn(0x7fd89c0d0b80 0x7fd89c0d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36230/0 (socket says 172.21.15.62:36230) 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd89affd640 1 --2- 172.21.15.62:0/1219603029 >> v2:172.21.15.174:3300/0 conn(0x7fd89c15f400 0x7fd89c0d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 -- 172.21.15.62:0/1219603029 >> v2:172.21.15.174:3300/0 conn(0x7fd89c15f400 msgr2=0x7fd89c0d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 --2- 172.21.15.62:0/1219603029 >> v2:172.21.15.174:3300/0 conn(0x7fd89c15f400 0x7fd89c0d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 -- 172.21.15.62:0/1219603029 >> v1:172.21.15.62:6789/0 conn(0x7fd89c0d0ff0 legacy=0x7fd89c0b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.855+0000 7fd893fff640 1 -- 172.21.15.62:0/1219603029 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c0d00a0 con 0x7fd89c0d0b80 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd893fff640 1 --2- 172.21.15.62:0/1219603029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0b80 0x7fd89c0d3740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd89c08e350 tx=0x7fd88402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57e42941293c223 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd89a7fc640 1 -- 172.21.15.62:0/1219603029 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884037070 con 0x7fd89c0d0b80 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd89a7fc640 1 -- 172.21.15.62:0/1219603029 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd884032440 con 0x7fd89c0d0b80 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd89a7fc640 1 -- 172.21.15.62:0/1219603029 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884032810 con 0x7fd89c0d0b80 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1219603029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0b80 msgr2=0x7fd89c0d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1219603029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0b80 0x7fd89c0d3740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd89c08e350 tx=0x7fd88402a1a0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1219603029 shutdown_connections 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1219603029 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0b80 0x7fd89c0d3740 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1219603029 >> v2:172.21.15.174:3300/0 conn(0x7fd89c15f400 0x7fd89c0d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1219603029 >> 172.21.15.62:0/1219603029 conn(0x7fd89c09e510 msgr2=0x7fd89c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:23.858 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1219603029 shutdown_connections 2024-04-26T22:28:23.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.856+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1219603029 wait complete. 2024-04-26T22:28:23.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.857+0000 7fd8a19c1640 1 Processor -- start 2024-04-26T22:28:23.859 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 -- start start 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 0x7fd89c0ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c15f400 0x7fd89c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c0cfd50 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c0d0660 con 0x7fd89c15f400 2024-04-26T22:28:23.861 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd8a19c1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c15f7e0 con 0x7fd89c0d0b80 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd893fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd893fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36240/0 (socket says 172.21.15.62:36240) 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd893fff640 1 -- 172.21.15.62:0/1883634266 learned_addr learned my addr 172.21.15.62:0/1883634266 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd89b7fe640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c15f400 0x7fd89c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.858+0000 7fd89affd640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 0x7fd89c0ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 -- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 msgr2=0x7fd89c0ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 0x7fd89c0ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 -- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c15f400 msgr2=0x7fd89c0cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c15f400 0x7fd89c0cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 -- 172.21.15.62:0/1883634266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c0cd040 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd89affd640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 0x7fd89c0ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd893fff640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd8840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884037040 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd884042070 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c0cd320 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884030bd0 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.859+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c160ed0 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.860+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c0b3db0 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.861+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd884047460 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.861+0000 7fd898ff9640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 0x7fd87c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.861+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd8840ac600 con 0x7fd89c0d0ff0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.861+0000 7fd89affd640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 0x7fd87c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:23.863 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.862+0000 7fd89affd640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 0x7fd87c06ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd88c006fd0 tx=0x7fd88c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:23.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:23.865+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd89c0b3db0 con 0x7fd89c0d0ff0 2024-04-26T22:28:24.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.093+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd89c0693e0 con 0x7fd89c0d0ff0 2024-04-26T22:28:24.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.093+0000 7fd898ff9640 1 -- 172.21.15.62:0/1883634266 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5184 (secure 0 0 0) 0x7fd884032e20 con 0x7fd89c0d0ff0 2024-04-26T22:28:24.094 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:24.094 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:01.910880+0000","modified":"2024-04-26T22:28:04.932395+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4837},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4837":{"gid":4837,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.62:6834/710033080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":710033080},{"type":"v1","addr":"172.21.15.62:6834","nonce":710033080}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4837,"qdb_cluster":[4837]},"id":2}]} 2024-04-26T22:28:24.094 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 17 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 msgr2=0x7fd87c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 0x7fd87c06ee40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd88c006fd0 tx=0x7fd88c008040 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 msgr2=0x7fd89c0cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd884000c00 tx=0x7fd8840046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 shutdown_connections 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd87c06c980 0x7fd87c06ee40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c15f400 0x7fd89c0cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd89c0d0ff0 0x7fd89c0cc4d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 --2- 172.21.15.62:0/1883634266 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd89c0d0b80 0x7fd89c0ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.095+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 >> 172.21.15.62:0/1883634266 conn(0x7fd89c09e510 msgr2=0x7fd89c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.096 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.096+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 shutdown_connections 2024-04-26T22:28:24.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.096+0000 7fd8a19c1640 1 -- 172.21.15.62:0/1883634266 wait complete. 2024-04-26T22:28:24.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.208+0000 7fa96fc2a640 1 Processor -- start 2024-04-26T22:28:24.208 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.208+0000 7fa96fc2a640 1 -- start start 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96fc2a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa96806ea40 0x7fa9680d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96fc2a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa9680cf830 0x7fa9680cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96fc2a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9680d0180 con 0x7fa968162e40 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96fc2a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fa968163220 con 0x7fa96806ea40 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96fc2a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa9680d0eb0 con 0x7fa9680cf830 2024-04-26T22:28:24.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa96806ea40 0x7fa9680d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fa96806ea40 0x7fa9680d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36250/0 (socket says 172.21.15.62:36250) 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 -- 172.21.15.62:0/1624496495 learned_addr learned my addr 172.21.15.62:0/1624496495 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96e1c7640 1 --2- 172.21.15.62:0/1624496495 >> v2:172.21.15.174:3300/0 conn(0x7fa9680cf830 0x7fa9680cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 -- 172.21.15.62:0/1624496495 >> v2:172.21.15.174:3300/0 conn(0x7fa9680cf830 msgr2=0x7fa9680cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 --2- 172.21.15.62:0/1624496495 >> v2:172.21.15.174:3300/0 conn(0x7fa9680cf830 0x7fa9680cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 -- 172.21.15.62:0/1624496495 >> v1:172.21.15.174:6789/0 conn(0x7fa968162e40 legacy=0x7fa9680b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.209+0000 7fa96d1c5640 1 -- 172.21.15.62:0/1624496495 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680b1d70 con 0x7fa96806ea40 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96d1c5640 1 --2- 172.21.15.62:0/1624496495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96806ea40 0x7fa9680d1340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa9680737a0 tx=0x7fa95802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc61c32bbb50f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96c9c4640 1 -- 172.21.15.62:0/1624496495 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958037070 con 0x7fa96806ea40 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96c9c4640 1 -- 172.21.15.62:0/1624496495 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa958032440 con 0x7fa96806ea40 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96c9c4640 1 -- 172.21.15.62:0/1624496495 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958032810 con 0x7fa96806ea40 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96fc2a640 1 -- 172.21.15.62:0/1624496495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96806ea40 msgr2=0x7fa9680d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.210+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/1624496495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96806ea40 0x7fa9680d1340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa9680737a0 tx=0x7fa95802a1e0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 -- 172.21.15.62:0/1624496495 shutdown_connections 2024-04-26T22:28:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/1624496495 >> v2:172.21.15.174:3300/0 conn(0x7fa9680cf830 0x7fa9680cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/1624496495 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa96806ea40 0x7fa9680d1340 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 -- 172.21.15.62:0/1624496495 >> 172.21.15.62:0/1624496495 conn(0x7fa96809ba50 msgr2=0x7fa96809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 -- 172.21.15.62:0/1624496495 shutdown_connections 2024-04-26T22:28:24.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.211+0000 7fa96fc2a640 1 -- 172.21.15.62:0/1624496495 wait complete. 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.212+0000 7fa96fc2a640 1 Processor -- start 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.212+0000 7fa96fc2a640 1 -- start start 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.212+0000 7fa96fc2a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa96806ea40 0x7fa9681f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.212+0000 7fa96fc2a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa968162e40 0x7fa96815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680d1f20 con 0x7fa9680cf830 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9680d1c40 con 0x7fa968162e40 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa9680ce010 con 0x7fa96806ea40 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36254/0 (socket says 172.21.15.62:36254) 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 -- 172.21.15.62:0/2571651704 learned_addr learned my addr 172.21.15.62:0/2571651704 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 -- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa96806ea40 msgr2=0x7fa9681f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa96806ea40 0x7fa9681f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.216 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 -- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa968162e40 msgr2=0x7fa96815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa968162e40 0x7fa96815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 -- 172.21.15.62:0/2571651704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9680b5b90 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96d1c5640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa95802a220 tx=0x7fa9580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958041020 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9580306c0 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958037040 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9680b5e70 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.213+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9680b6430 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.214+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96806a360 con 0x7fa9680cf830 2024-04-26T22:28:24.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.215+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa958044020 con 0x7fa9680cf830 2024-04-26T22:28:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.215+0000 7fa95e7fc640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 0x7fa94c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.215+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa9580ac3d0 con 0x7fa9680cf830 2024-04-26T22:28:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.215+0000 7fa96d9c6640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 0x7fa94c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.216+0000 7fa96d9c6640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 0x7fa94c06ee40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa960000f60 tx=0x7fa9600079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.219+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa95804b030 con 0x7fa9680cf830 2024-04-26T22:28:24.405 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.405+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96809c1b0 con 0x7fa9680cf830 2024-04-26T22:28:24.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.406+0000 7fa95e7fc640 1 -- 172.21.15.62:0/2571651704 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16669 (secure 0 0 0) 0x7fa95807a810 con 0x7fa9680cf830 2024-04-26T22:28:24.406 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:24.407 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":31,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:03.546754+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:00.329554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:00.858734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:24.408 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 msgr2=0x7fa94c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 0x7fa94c06ee40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa960000f60 tx=0x7fa9600079d0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 msgr2=0x7fa9680b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa95802a220 tx=0x7fa9580046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 shutdown_connections 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa94c06c980 0x7fa94c06ee40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa968162e40 0x7fa96815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9680cf830 0x7fa9680b10b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 --2- 172.21.15.62:0/2571651704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa96806ea40 0x7fa9681f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 >> 172.21.15.62:0/2571651704 conn(0x7fa96809ba50 msgr2=0x7fa96815d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 shutdown_connections 2024-04-26T22:28:24.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.408+0000 7fa96fc2a640 1 -- 172.21.15.62:0/2571651704 wait complete. 2024-04-26T22:28:24.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T22:28:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.520+0000 7fbd3c1e4640 1 Processor -- start 2024-04-26T22:28:24.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.521+0000 7fbd3c1e4640 1 -- start start 2024-04-26T22:28:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.521+0000 7fbd3c1e4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd340d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3c1e4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340b1d70 con 0x7fbd340cf830 2024-04-26T22:28:24.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3c1e4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd340d1960 con 0x7fbd34162e40 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3c1e4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbd34163220 con 0x7fbd3406ea40 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3a781640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fbd340cf830 0x7fbd340ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57712/0 (socket says 172.21.15.62:57712) 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3a781640 1 -- 172.21.15.62:0/3762289552 learned_addr learned my addr 172.21.15.62:0/3762289552 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3977f640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd340d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 459389362 0 0) 0x7fbd340b1d70 con 0x7fbd340cf830 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd20003520 con 0x7fbd340cf830 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1915120913 0 0) 0x7fbd20003520 con 0x7fbd340cf830 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 msgr2=0x7fbd340d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd340d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 >> v1:172.21.15.174:6789/0 conn(0x7fbd34162e40 legacy=0x7fbd340b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd340d1c40 con 0x7fbd340cf830 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fbd2c0019f0 con 0x7fbd340cf830 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 >> v1:172.21.15.62:6789/0 conn(0x7fbd340cf830 legacy=0x7fbd340ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd200039e0 0x7fbd20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd20004800 0x7fbd20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd20005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2000baf0 con 0x7fbd20004800 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2000bca0 con 0x7fbd200039e0 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2000bea0 con 0x7fbd3406ea40 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.522+0000 7fbd3977f640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd20004800 0x7fbd20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd39f80640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd200039e0 0x7fbd20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3a781640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd20005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 -- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 msgr2=0x7fbd20005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd20005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 -- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd200039e0 msgr2=0x7fbd20003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd200039e0 0x7fbd20003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 -- 172.21.15.62:0/3762289552 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd20003520 con 0x7fbd20004800 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3a781640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd3406ea40 0x7fbd20005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.523+0000 7fbd3977f640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd20004800 0x7fbd20004c80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009ac0 tx=0x7fbd20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400c74ece3701a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24036070 con 0x7fbd20004800 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24031400 con 0x7fbd20004800 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd38f7e640 1 -- 172.21.15.62:0/3762289552 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24009690 con 0x7fbd20004800 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/3762289552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd20004800 msgr2=0x7fbd20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/3762289552 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd20004800 0x7fbd20004c80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009ac0 tx=0x7fbd20005830 comp rx=0 tx=0).stop 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3a781640 1 -- 172.21.15.62:0/3762289552 reap_dead start 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/3762289552 shutdown_connections 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/3762289552 >> 172.21.15.62:0/3762289552 conn(0x7fbd3409ba50 msgr2=0x7fbd3409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/3762289552 shutdown_connections 2024-04-26T22:28:24.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.524+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/3762289552 wait complete. 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.525+0000 7fbd3c1e4640 1 Processor -- start 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.525+0000 7fbd3c1e4640 1 -- start start 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd20004800 0x7fbd3415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 0x7fbd3415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340d1340 con 0x7fbd341f4340 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd340d0eb0 con 0x7fbd3415a9c0 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3c1e4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbd340d14c0 con 0x7fbd20004800 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3a781640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3a781640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36270/0 (socket says 172.21.15.62:36270) 2024-04-26T22:28:24.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3a781640 1 -- 172.21.15.62:0/764378989 learned_addr learned my addr 172.21.15.62:0/764378989 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd39f80640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd20004800 0x7fbd3415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.526+0000 7fbd3977f640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 0x7fbd3415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 -- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd20004800 msgr2=0x7fbd3415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd20004800 0x7fbd3415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 -- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 msgr2=0x7fbd3415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 0x7fbd3415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 -- 172.21.15.62:0/764378989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd34159920 con 0x7fbd341f4340 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3977f640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 0x7fbd3415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3a781640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd340737a0 tx=0x7fbd2c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00eda0 con 0x7fbd341f4340 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c018070 con 0x7fbd341f4340 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c013660 con 0x7fbd341f4340 2024-04-26T22:28:24.532 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd34159c00 con 0x7fbd341f4340 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.527+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3415a1c0 con 0x7fbd341f4340 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.528+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3406a360 con 0x7fbd341f4340 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.528+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fbd2c013800 con 0x7fbd341f4340 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.529+0000 7fbd2a7fc640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 0x7fbd1806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.529+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbd2c08bf80 con 0x7fbd341f4340 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.529+0000 7fbd39f80640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 0x7fbd1806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.530+0000 7fbd39f80640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 0x7fbd1806ee70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd300017e0 tx=0x7fbd30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.534+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd2c05a3c0 con 0x7fbd341f4340 2024-04-26T22:28:24.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.758+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbd3415a530 con 0x7fbd341f4340 2024-04-26T22:28:24.759 INFO:tasks.ceph.mon.a.smithi062.stderr:2024-04-26T22:28:24.759+0000 7facc72cb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T22:28:24.760 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.760+0000 7fbd2a7fc640 1 -- 172.21.15.62:0/764378989 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fbd2c059cc0 con 0x7fbd341f4340 2024-04-26T22:28:24.760 INFO:teuthology.orchestra.run.smithi062.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T22:28:24.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 msgr2=0x7fbd1806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 0x7fbd1806ee70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd300017e0 tx=0x7fbd30009040 comp rx=0 tx=0).stop 2024-04-26T22:28:24.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 msgr2=0x7fbd340c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd340737a0 tx=0x7fbd2c00e530 comp rx=0 tx=0).stop 2024-04-26T22:28:24.763 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 shutdown_connections 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fbd1806c9b0 0x7fbd1806ee70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd341f4340 0x7fbd340c8d80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd3415a9c0 0x7fbd3415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 --2- 172.21.15.62:0/764378989 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbd20004800 0x7fbd3415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 >> 172.21.15.62:0/764378989 conn(0x7fbd3409ba50 msgr2=0x7fbd340b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 shutdown_connections 2024-04-26T22:28:24.764 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.763+0000 7fbd3c1e4640 1 -- 172.21.15.62:0/764378989 wait complete. 2024-04-26T22:28:24.766 INFO:tasks.ceph.mds.a.smithi062.stderr: -4328> 2024-04-26T22:28:07.349+0000 7fc11b108640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:28:24.774 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T22:28:24.796 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-26T22:28:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.877+0000 7f18b4c9c640 1 Processor -- start 2024-04-26T22:28:24.877 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.877+0000 7f18b4c9c640 1 -- start start 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18b4c9c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f18b00cff80 0x7f18b00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18b4c9c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18b00d1fe0 0x7f18b00d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18b4c9c640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b00cd640 con 0x7f18b00b2290 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18b4c9c640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f18b00cd920 con 0x7f18b00d1fe0 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18b4c9c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f18b00cdc00 con 0x7f18b00cff80 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18aed9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f18b00d1fe0 0x7f18b00d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.879 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18ae59c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f18b00b2290 0x7f18b00cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57718/0 (socket says 172.21.15.62:57718) 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18ae59c640 1 -- 172.21.15.62:0/1605302568 learned_addr learned my addr 172.21.15.62:0/1605302568 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18a5d9b640 1 --2- 172.21.15.62:0/1605302568 >> v2:172.21.15.174:3300/0 conn(0x7f18b00cff80 0x7f18b00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18aed9d640 1 -- 172.21.15.62:0/1605302568 >> v2:172.21.15.174:3300/0 conn(0x7f18b00cff80 msgr2=0x7f18b00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18aed9d640 1 --2- 172.21.15.62:0/1605302568 >> v2:172.21.15.174:3300/0 conn(0x7f18b00cff80 0x7f18b00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18aed9d640 1 -- 172.21.15.62:0/1605302568 >> v1:172.21.15.62:6789/0 conn(0x7f18b00b2290 legacy=0x7f18b00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18add9b640 1 -- 172.21.15.62:0/1605302568 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323813172 0 0) 0x7f18b00cd640 con 0x7f18b00b2290 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.878+0000 7f18aed9d640 1 -- 172.21.15.62:0/1605302568 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b00cdee0 con 0x7f18b00d1fe0 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.879+0000 7f18aed9d640 1 --2- 172.21.15.62:0/1605302568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00d1fe0 0x7f18b00d23f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f18b0073b10 tx=0x7f18a802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d087d419ca0c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.879+0000 7f18add9b640 1 -- 172.21.15.62:0/1605302568 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007740 con 0x7f18b00d1fe0 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.879+0000 7f18add9b640 1 -- 172.21.15.62:0/1605302568 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80078e0 con 0x7f18b00d1fe0 2024-04-26T22:28:24.880 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.879+0000 7f18add9b640 1 -- 172.21.15.62:0/1605302568 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8007cb0 con 0x7f18b00d1fe0 2024-04-26T22:28:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 -- 172.21.15.62:0/1605302568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00d1fe0 msgr2=0x7f18b00d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/1605302568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00d1fe0 0x7f18b00d23f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f18b0073b10 tx=0x7f18a802a680 comp rx=0 tx=0).stop 2024-04-26T22:28:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 -- 172.21.15.62:0/1605302568 shutdown_connections 2024-04-26T22:28:24.881 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/1605302568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00d1fe0 0x7f18b00d23f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/1605302568 >> v2:172.21.15.174:3300/0 conn(0x7f18b00cff80 0x7f18b00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 -- 172.21.15.62:0/1605302568 >> 172.21.15.62:0/1605302568 conn(0x7f18b00d2ff0 msgr2=0x7f18b00d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 -- 172.21.15.62:0/1605302568 shutdown_connections 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.880+0000 7f18b4c9c640 1 -- 172.21.15.62:0/1605302568 wait complete. 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.881+0000 7f18b4c9c640 1 Processor -- start 2024-04-26T22:28:24.882 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.881+0000 7f18b4c9c640 1 -- start start 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18b00b2290 0x7f18b00cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b00d1fe0 0x7f18b01f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00d2930 con 0x7f18b00cff80 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00b2670 con 0x7f18b00d1fe0 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18b4c9c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f18b00d2ab0 con 0x7f18b00b2290 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.884 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36308/0 (socket says 172.21.15.62:36308) 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 -- 172.21.15.62:0/2869312232 learned_addr learned my addr 172.21.15.62:0/2869312232 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18ae59c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18b00b2290 0x7f18b00cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18aed9d640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b00d1fe0 0x7f18b01f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 -- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18b00b2290 msgr2=0x7f18b00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18b00b2290 0x7f18b00cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 -- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b00d1fe0 msgr2=0x7f18b01f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:24.885 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b00d1fe0 0x7f18b01f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 -- 172.21.15.62:0/2869312232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b00b79a0 con 0x7f18b00cff80 2024-04-26T22:28:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.882+0000 7f18a5d9b640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9b0 tx=0x7f189800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.883+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898017070 con 0x7f18b00cff80 2024-04-26T22:28:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.883+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189800c9a0 con 0x7f18b00cff80 2024-04-26T22:28:24.886 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.883+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01f55f0 con 0x7f18b00cff80 2024-04-26T22:28:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.883+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800cd90 con 0x7f18b00cff80 2024-04-26T22:28:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.883+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b00b7c20 con 0x7f18b00cff80 2024-04-26T22:28:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.884+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b01f5b10 con 0x7f18b00cff80 2024-04-26T22:28:24.887 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.884+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f189801a070 con 0x7f18b00cff80 2024-04-26T22:28:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.884+0000 7f18a77fe640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 0x7f189006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.885+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f189808bc20 con 0x7f18b00cff80 2024-04-26T22:28:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.885+0000 7f18ae59c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 0x7f189006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.885+0000 7f18ae59c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 0x7f189006ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000f60 tx=0x7f18a0007b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:24.889 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:24.889+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18b01f5b10 con 0x7f18b00cff80 2024-04-26T22:28:25.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.112+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f18b00b8020 con 0x7f18b00cff80 2024-04-26T22:28:25.768 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.768+0000 7f18a77fe640 1 -- 172.21.15.62:0/2869312232 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f18b00b79a0 con 0x7f18b00cff80 2024-04-26T22:28:25.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 msgr2=0x7f189006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 0x7f189006ee40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000f60 tx=0x7f18a0007b70 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 msgr2=0x7f18b01f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f189800e9b0 tx=0x7f189800ee70 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 shutdown_connections 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f189006c980 0x7f189006ee40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b00d1fe0 0x7f18b01f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f18b00cff80 0x7f18b01f5d20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 --2- 172.21.15.62:0/2869312232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f18b00b2290 0x7f18b00cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 >> 172.21.15.62:0/2869312232 conn(0x7f18b00d2ff0 msgr2=0x7f18b006dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 shutdown_connections 2024-04-26T22:28:25.771 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.770+0000 7f18b4c9c640 1 -- 172.21.15.62:0/2869312232 wait complete. 2024-04-26T22:28:25.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T22:28:25.932 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.932+0000 7f752f2f8640 1 Processor -- start 2024-04-26T22:28:25.933 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.932+0000 7f752f2f8640 1 -- start start 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752f2f8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f752815f390 0x7f7528072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752f2f8640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f752815f770 con 0x7f75280b1e50 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752f2f8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75280d2ad0 con 0x7f752806f0d0 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752f2f8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f75280d2db0 con 0x7f752815f390 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752c893640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f75280b1e50 0x7f752806e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57728/0 (socket says 172.21.15.62:57728) 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752c893640 1 -- 172.21.15.62:0/108156721 learned_addr learned my addr 172.21.15.62:0/108156721 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f752d094640 1 --2- 172.21.15.62:0/108156721 >> v2:172.21.15.174:3300/0 conn(0x7f752815f390 0x7f7528072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 653543731 0 0) 0x7f752815f770 con 0x7f75280b1e50 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.933+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7514003520 con 0x7f75280b1e50 2024-04-26T22:28:25.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118547284 0 0) 0x7f75280d2ad0 con 0x7f752806f0d0 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f752815f770 con 0x7f752806f0d0 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f752d094640 1 -- 172.21.15.62:0/108156721 >> v1:172.21.15.174:6789/0 conn(0x7f752806f0d0 legacy=0x7f75281631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f752d094640 1 -- 172.21.15.62:0/108156721 >> v1:172.21.15.62:6789/0 conn(0x7f75280b1e50 legacy=0x7f752806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f752d094640 1 -- 172.21.15.62:0/108156721 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75280d0c60 con 0x7f752815f390 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.934+0000 7f752d094640 1 --2- 172.21.15.62:0/108156721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752815f390 0x7f7528072620 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75180097a0 tx=0x7f751802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988c07ef408c41c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518037070 con 0x7f752815f390 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7518032440 con 0x7f752815f390 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f751ffff640 1 -- 172.21.15.62:0/108156721 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7518032810 con 0x7f752815f390 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 -- 172.21.15.62:0/108156721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752815f390 msgr2=0x7f7528072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 --2- 172.21.15.62:0/108156721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752815f390 0x7f7528072620 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f75180097a0 tx=0x7f751802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 -- 172.21.15.62:0/108156721 shutdown_connections 2024-04-26T22:28:25.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 --2- 172.21.15.62:0/108156721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752815f390 0x7f7528072620 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 -- 172.21.15.62:0/108156721 >> 172.21.15.62:0/108156721 conn(0x7f75280d2f60 msgr2=0x7f75280d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 -- 172.21.15.62:0/108156721 shutdown_connections 2024-04-26T22:28:25.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.935+0000 7f752f2f8640 1 -- 172.21.15.62:0/108156721 wait complete. 2024-04-26T22:28:25.941 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.936+0000 7f752f2f8640 1 Processor -- start 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.936+0000 7f752f2f8640 1 -- start start 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 0x7f7528152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752815f390 0x7f75280cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f75280d1220 con 0x7f75280b1e50 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75280d2780 con 0x7f752806f0d0 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752f2f8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f75280cd5b0 con 0x7f752815f390 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752c893640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752c893640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36322/0 (socket says 172.21.15.62:36322) 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752c893640 1 -- 172.21.15.62:0/420679442 learned_addr learned my addr 172.21.15.62:0/420679442 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:25.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752d094640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 0x7f7528152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.937+0000 7f752d895640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752815f390 0x7f75280cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 -- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752815f390 msgr2=0x7f75280cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752815f390 0x7f75280cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 -- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 msgr2=0x7f7528152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 0x7f7528152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 -- 172.21.15.62:0/420679442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75281528a0 con 0x7f75280b1e50 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752d094640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 0x7f7528152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:25.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752c893640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f752400c500 tx=0x7f752400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524017070 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524005ce0 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752815fb40 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75240072f0 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.938+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7528160100 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.939+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752806a360 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.939+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f752401a070 con 0x7f75280b1e50 2024-04-26T22:28:25.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.940+0000 7f751dffb640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 0x7f750c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.940+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f752408ae60 con 0x7f75280b1e50 2024-04-26T22:28:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.940+0000 7f752d094640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 0x7f750c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.941+0000 7f752d094640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 0x7f750c06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7518034000 tx=0x7f751802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:25.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:25.944+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7524058c80 con 0x7f75280b1e50 2024-04-26T22:28:26.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.130+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75280b8ee0 con 0x7f75280b1e50 2024-04-26T22:28:26.770 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.769+0000 7f751dffb640 1 -- 172.21.15.62:0/420679442 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v33) ==== 181+0+0 (secure 0 0 0) 0x7f752405cf80 con 0x7f75280b1e50 2024-04-26T22:28:26.770 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' removed 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 msgr2=0x7f750c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 0x7f750c06ee40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7518034000 tx=0x7f751802e000 comp rx=0 tx=0).stop 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 msgr2=0x7f75280c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f752400c500 tx=0x7f752400c9c0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 shutdown_connections 2024-04-26T22:28:26.773 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f750c06c980 0x7f750c06ee40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752815f390 0x7f75280cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f75280b1e50 0x7f75280c9530 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 --2- 172.21.15.62:0/420679442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752806f0d0 0x7f7528152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 >> 172.21.15.62:0/420679442 conn(0x7f75280d2f60 msgr2=0x7f75280cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 shutdown_connections 2024-04-26T22:28:26.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.773+0000 7f752f2f8640 1 -- 172.21.15.62:0/420679442 wait complete. 2024-04-26T22:28:26.787 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T22:28:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.900+0000 7f0a91d71640 1 Processor -- start 2024-04-26T22:28:26.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.900+0000 7f0a91d71640 1 -- start start 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a91d71640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a8c0d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a91d71640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8c0b50f0 con 0x7f0a8c0d4010 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a91d71640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a8c0d0a80 con 0x7f0a8c0d5640 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a91d71640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c0b5420 con 0x7f0a8c15f5d0 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a8affd640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0a8c0d4010 0x7f0a8c0d4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57736/0 (socket says 172.21.15.62:57736) 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.901+0000 7f0a8affd640 1 -- 172.21.15.62:0/3546954927 learned_addr learned my addr 172.21.15.62:0/3546954927 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a8c0d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292115497 0 0) 0x7f0a8c0b50f0 con 0x7f0a8c0d4010 2024-04-26T22:28:26.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a78003520 con 0x7f0a8c0d4010 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 613725808 0 0) 0x7f0a78003520 con 0x7f0a8c0d4010 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 msgr2=0x7f0a8c0d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a8c0d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 >> v1:172.21.15.174:6789/0 conn(0x7f0a8c0d5640 legacy=0x7f0a8c0b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c0d0d60 con 0x7f0a8c0d4010 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f0a74001670 con 0x7f0a8c0d4010 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 >> v1:172.21.15.62:6789/0 conn(0x7f0a8c0d4010 legacy=0x7f0a8c0d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a780039e0 0x7f0a78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a78004800 0x7f0a78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c0b5420 con 0x7f0a78004800 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7800bb00 con 0x7f0a780039e0 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7800bd00 con 0x7f0a8c15f5d0 2024-04-26T22:28:26.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a78004800 0x7f0a78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a780039e0 0x7f0a78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8bfff640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 -- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 msgr2=0x7f0a780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c15f5d0 0x7f0a780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 -- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a780039e0 msgr2=0x7f0a78003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a780039e0 0x7f0a78003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8affd640 1 -- 172.21.15.62:0/3546954927 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a78003520 con 0x7f0a78004800 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.902+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a780039e0 0x7f0a78003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.903+0000 7f0a8affd640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a78004800 0x7f0a78004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001eb0 tx=0x7f0a78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c791c30a46f3dc16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.903+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74007d30 con 0x7f0a78004800 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.903+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a74005af0 con 0x7f0a78004800 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.903+0000 7f0a8a7fc640 1 -- 172.21.15.62:0/3546954927 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74007070 con 0x7f0a78004800 2024-04-26T22:28:26.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 -- 172.21.15.62:0/3546954927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a78004800 msgr2=0x7f0a78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 --2- 172.21.15.62:0/3546954927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a78004800 0x7f0a78004c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001eb0 tx=0x7f0a78005830 comp rx=0 tx=0).stop 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a8bfff640 1 -- 172.21.15.62:0/3546954927 reap_dead start 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 -- 172.21.15.62:0/3546954927 shutdown_connections 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 -- 172.21.15.62:0/3546954927 >> 172.21.15.62:0/3546954927 conn(0x7f0a8c08ebd0 msgr2=0x7f0a8c09e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 -- 172.21.15.62:0/3546954927 shutdown_connections 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.904+0000 7f0a91d71640 1 -- 172.21.15.62:0/3546954927 wait complete. 2024-04-26T22:28:26.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.905+0000 7f0a91d71640 1 Processor -- start 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.905+0000 7f0a91d71640 1 -- start start 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78004800 0x7f0a8c0cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c0c9160 0x7f0a8c1f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c0d34f0 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c15f9b0 con 0x7f0a78004800 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a91d71640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8c0d3ad0 con 0x7f0a8c0c9160 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a8affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36344/0 (socket says 172.21.15.62:36344) 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a8affd640 1 -- 172.21.15.62:0/247899106 learned_addr learned my addr 172.21.15.62:0/247899106 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:26.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a8bfff640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c0c9160 0x7f0a8c1f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.906+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78004800 0x7f0a8c0cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 -- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c0c9160 msgr2=0x7f0a8c1f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c0c9160 0x7f0a8c1f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 -- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78004800 msgr2=0x7f0a8c0cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78004800 0x7f0a8c0cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 -- 172.21.15.62:0/247899106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c0c9950 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a8affd640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c2d0 tx=0x7f0a74001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74001630 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a74042070 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74007270 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c15fcd0 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.907+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c160290 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.909+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c06a360 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.911+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0a74007410 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.911+0000 7f0a83fff640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 0x7f0a6c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:26.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.911+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5357+0+0 (secure 0 0 0) 0x7f0a740ab7a0 con 0x7f0a8c0cb5d0 2024-04-26T22:28:26.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.912+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 0x7f0a6c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:26.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.913+0000 7f0a8b7fe640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 0x7f0a6c06ee70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c15a7e0 tx=0x7f0a7c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:26.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:26.917+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a74079840 con 0x7f0a8c0cb5d0 2024-04-26T22:28:27.105 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.105+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a8c0cae70 con 0x7f0a8c0cb5d0 2024-04-26T22:28:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.774+0000 7f0a83fff640 1 -- 172.21.15.62:0/247899106 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v34) ==== 169+0+0 (secure 0 0 0) 0x7f0a7407db40 con 0x7f0a8c0cb5d0 2024-04-26T22:28:27.775 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' removed 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 msgr2=0x7f0a6c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 0x7f0a6c06ee70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c15a7e0 tx=0x7f0a7c00b040 comp rx=0 tx=0).stop 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 msgr2=0x7f0a8c0c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400c2d0 tx=0x7f0a74001a40 comp rx=0 tx=0).stop 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 shutdown_connections 2024-04-26T22:28:27.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0a6c06c9b0 0x7f0a6c06ee70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0a8c0c9160 0x7f0a8c1f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0a8c0cb5d0 0x7f0a8c0c8c20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 --2- 172.21.15.62:0/247899106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a78004800 0x7f0a8c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 >> 172.21.15.62:0/247899106 conn(0x7f0a8c08ebd0 msgr2=0x7f0a8c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 shutdown_connections 2024-04-26T22:28:27.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.779+0000 7f0a91d71640 1 -- 172.21.15.62:0/247899106 wait complete. 2024-04-26T22:28:27.794 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T22:28:27.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.903+0000 7fd4c796a640 1 Processor -- start 2024-04-26T22:28:27.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.903+0000 7fd4c796a640 1 -- start start 2024-04-26T22:28:27.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c796a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd4c015c450 0x7fd4c006df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:27.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c796a640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c015c830 con 0x7fd4c00cc870 2024-04-26T22:28:27.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c796a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c015d600 con 0x7fd4c00cbcd0 2024-04-26T22:28:27.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c796a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c015d8e0 con 0x7fd4c015c450 2024-04-26T22:28:27.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5706640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd4c015c450 0x7fd4c006df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:27.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5f07640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fd4c00cc870 0x7fd4c00729f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57742/0 (socket says 172.21.15.62:57742) 2024-04-26T22:28:27.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5f07640 1 -- 172.21.15.62:0/3484098397 learned_addr learned my addr 172.21.15.62:0/3484098397 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:27.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5706640 1 -- 172.21.15.62:0/3484098397 >> v1:172.21.15.174:6789/0 conn(0x7fd4c00cbcd0 legacy=0x7fd4c00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:27.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5706640 1 -- 172.21.15.62:0/3484098397 >> v1:172.21.15.62:6789/0 conn(0x7fd4c00cc870 legacy=0x7fd4c00729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.904+0000 7fd4c5706640 1 -- 172.21.15.62:0/3484098397 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c006f830 con 0x7fd4c015c450 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.905+0000 7fd4c5706640 1 --2- 172.21.15.62:0/3484098397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c006df90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097a0 tx=0x7fd4bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7426f69e837824b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.905+0000 7fd4b7fff640 1 -- 172.21.15.62:0/3484098397 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc037070 con 0x7fd4c015c450 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.905+0000 7fd4b7fff640 1 -- 172.21.15.62:0/3484098397 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02ae50 con 0x7fd4c015c450 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.905+0000 7fd4b7fff640 1 -- 172.21.15.62:0/3484098397 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc032660 con 0x7fd4c015c450 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 -- 172.21.15.62:0/3484098397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 msgr2=0x7fd4c006df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 --2- 172.21.15.62:0/3484098397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c006df90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097a0 tx=0x7fd4bc02a140 comp rx=0 tx=0).stop 2024-04-26T22:28:27.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 -- 172.21.15.62:0/3484098397 shutdown_connections 2024-04-26T22:28:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 --2- 172.21.15.62:0/3484098397 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c006df90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 -- 172.21.15.62:0/3484098397 >> 172.21.15.62:0/3484098397 conn(0x7fd4c00d0d70 msgr2=0x7fd4c00d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 -- 172.21.15.62:0/3484098397 shutdown_connections 2024-04-26T22:28:27.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.906+0000 7fd4c796a640 1 -- 172.21.15.62:0/3484098397 wait complete. 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.907+0000 7fd4c796a640 1 Processor -- start 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.907+0000 7fd4c796a640 1 -- start start 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 0x7fd4c0156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c00cc870 0x7fd4c00c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0073280 con 0x7fd4c015c450 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0073100 con 0x7fd4c00cbcd0 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c0073400 con 0x7fd4c00cc870 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36360/0 (socket says 172.21.15.62:36360) 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 -- 172.21.15.62:0/1722201360 learned_addr learned my addr 172.21.15.62:0/1722201360 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5706640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 0x7fd4c0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c4f05640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c00cc870 0x7fd4c00c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:27.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 -- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c00cc870 msgr2=0x7fd4c00c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c00cc870 0x7fd4c00c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 -- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 msgr2=0x7fd4c0156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 0x7fd4c0156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 -- 172.21.15.62:0/1722201360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c01f79d0 con 0x7fd4c015c450 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5706640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 0x7fd4c0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c5f07640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800cbc0 tx=0x7fd4b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8017070 con 0x7fd4c015c450 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b800ad40 con 0x7fd4c015c450 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c00b57d0 con 0x7fd4c015c450 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.908+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8012600 con 0x7fd4c015c450 2024-04-26T22:28:27.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.909+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c0156670 con 0x7fd4c015c450 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.910+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c006a360 con 0x7fd4c015c450 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.910+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd4b801a070 con 0x7fd4c015c450 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.910+0000 7fd4b5ffb640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 0x7fd4a406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.910+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fd4b808b490 con 0x7fd4c015c450 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.910+0000 7fd4c5706640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 0x7fd4a406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:27.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.911+0000 7fd4c5706640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 0x7fd4a406ee70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc034000 tx=0x7fd4bc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:27.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:27.914+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4b8059610 con 0x7fd4c015c450 2024-04-26T22:28:28.101 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.101+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd4c00cbab0 con 0x7fd4c015c450 2024-04-26T22:28:28.103 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.103+0000 7fd4b5ffb640 1 -- 172.21.15.62:0/1722201360 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fd4b805d910 con 0x7fd4c015c450 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.105+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 msgr2=0x7fd4a406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.105+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 0x7fd4a406ee70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc034000 tx=0x7fd4bc02e000 comp rx=0 tx=0).stop 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.105+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 msgr2=0x7fd4c00ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.105+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800cbc0 tx=0x7fd4b800a430 comp rx=0 tx=0).stop 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 shutdown_connections 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd4a406c9b0 0x7fd4a406ee70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.106 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd4c015c450 0x7fd4c00ca290 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4c00cc870 0x7fd4c00c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 --2- 172.21.15.62:0/1722201360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4c00cbcd0 0x7fd4c0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 >> 172.21.15.62:0/1722201360 conn(0x7fd4c00d0d70 msgr2=0x7fd4c006df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:28.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 shutdown_connections 2024-04-26T22:28:28.107 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.106+0000 7fd4c796a640 1 -- 172.21.15.62:0/1722201360 wait complete. 2024-04-26T22:28:28.117 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2024-04-26T22:28:28.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.217+0000 7fc1a15f0640 1 Processor -- start 2024-04-26T22:28:28.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.217+0000 7fc1a15f0640 1 -- start start 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc19c159a90 0x7fc19c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc19c160150 0x7fc19c15c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c160580 con 0x7fc19c160150 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c072e30 con 0x7fc19c159a90 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc1a15f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c073110 con 0x7fc19c070130 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc19c160150 0x7fc19c15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fc19c160150 0x7fc19c15c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36362/0 (socket says 172.21.15.62:36362) 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 -- 172.21.15.62:0/3236866500 learned_addr learned my addr 172.21.15.62:0/3236866500 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19a7fc640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19affd640 1 --2- 172.21.15.62:0/3236866500 >> v2:172.21.15.174:3300/0 conn(0x7fc19c159a90 0x7fc19c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 -- 172.21.15.62:0/3236866500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 msgr2=0x7fc19c15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c15fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 -- 172.21.15.62:0/3236866500 >> v2:172.21.15.174:3300/0 conn(0x7fc19c159a90 msgr2=0x7fc19c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:28.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 --2- 172.21.15.62:0/3236866500 >> v2:172.21.15.174:3300/0 conn(0x7fc19c159a90 0x7fc19c06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.218+0000 7fc19b7fe640 1 -- 172.21.15.62:0/3236866500 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c0733f0 con 0x7fc19c160150 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.219+0000 7fc19b7fe640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c160150 0x7fc19c15c620 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc19c073040 tx=0x7fc18c02a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb9bb7996a0b5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.219+0000 7fc199ffb640 1 -- 172.21.15.62:0/3236866500 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c036070 con 0x7fc19c160150 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.219+0000 7fc199ffb640 1 -- 172.21.15.62:0/3236866500 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c031600 con 0x7fc19c160150 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.219+0000 7fc199ffb640 1 -- 172.21.15.62:0/3236866500 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0319d0 con 0x7fc19c160150 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 -- 172.21.15.62:0/3236866500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c160150 msgr2=0x7fc19c15c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c160150 0x7fc19c15c620 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc19c073040 tx=0x7fc18c02a560 comp rx=0 tx=0).stop 2024-04-26T22:28:28.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 -- 172.21.15.62:0/3236866500 shutdown_connections 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c160150 0x7fc19c15c620 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/3236866500 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c15fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/3236866500 >> v2:172.21.15.174:3300/0 conn(0x7fc19c159a90 0x7fc19c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 -- 172.21.15.62:0/3236866500 >> 172.21.15.62:0/3236866500 conn(0x7fc19c0d0d50 msgr2=0x7fc19c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 -- 172.21.15.62:0/3236866500 shutdown_connections 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.220+0000 7fc1a15f0640 1 -- 172.21.15.62:0/3236866500 wait complete. 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.221+0000 7fc1a15f0640 1 Processor -- start 2024-04-26T22:28:28.222 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.221+0000 7fc1a15f0640 1 -- start start 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc19c160150 0x7fc19c156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c159e70 con 0x7fc19c159a90 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1609d0 con 0x7fc19c160150 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc1a15f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c159ff0 con 0x7fc19c070130 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36372/0 (socket says 172.21.15.62:36372) 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 -- 172.21.15.62:0/2889594435 learned_addr learned my addr 172.21.15.62:0/2889594435 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:28.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19affd640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19b7fe640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc19c160150 0x7fc19c156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 -- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 msgr2=0x7fc19c148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 -- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc19c160150 msgr2=0x7fc19c156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc19c160150 0x7fc19c156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.222+0000 7fc19a7fc640 1 -- 172.21.15.62:0/2889594435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c14baa0 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc19a7fc640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d800 tx=0x7fc18400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184018030 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1840096a0 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c14bd80 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184009870 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.223+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c14c340 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.224+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c06a360 con 0x7fc19c159a90 2024-04-26T22:28:28.225 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.224+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc18401b070 con 0x7fc19c159a90 2024-04-26T22:28:28.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.225+0000 7fc1937fe640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 0x7fc17c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.225+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fc18408ba60 con 0x7fc19c159a90 2024-04-26T22:28:28.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.225+0000 7fc19affd640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 0x7fc17c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.226 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.225+0000 7fc19affd640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 0x7fc17c06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.229 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.229+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc184059ac0 con 0x7fc19c159a90 2024-04-26T22:28:28.486 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.486+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) -- 0x7fc19c1576f0 con 0x7fc19c159a90 2024-04-26T22:28:28.775 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.776+0000 7fc1937fe640 1 -- 172.21.15.62:0/2889594435 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v91) ==== 181+0+0 (secure 0 0 0) 0x7fc18405ddc0 con 0x7fc19c159a90 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 msgr2=0x7fc17c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 0x7fc17c06ee40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc194006fd0 tx=0x7fc194008040 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 msgr2=0x7fc19c155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc18400d800 tx=0x7fc18400dcc0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 shutdown_connections 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc17c06c980 0x7fc17c06ee40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc19c160150 0x7fc19c156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc19c159a90 0x7fc19c155d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 --2- 172.21.15.62:0/2889594435 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc19c070130 0x7fc19c148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.779 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 >> 172.21.15.62:0/2889594435 conn(0x7fc19c0d0d50 msgr2=0x7fc19c0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 shutdown_connections 2024-04-26T22:28:28.780 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.779+0000 7fc1a15f0640 1 -- 172.21.15.62:0/2889594435 wait complete. 2024-04-26T22:28:28.790 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2024-04-26T22:28:28.791 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2024-04-26T22:28:28.791 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2024-04-26T22:28:28.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.892+0000 7f0f57dec640 1 Processor -- start 2024-04-26T22:28:28.892 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.892+0000 7f0f57dec640 1 -- start start 2024-04-26T22:28:28.893 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f57dec640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f500b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f57dec640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f500d3de0 con 0x7f0f5015f380 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f57dec640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f500d40c0 con 0x7f0f500d22e0 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f57dec640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0f500d43a0 con 0x7f0f5015f760 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f55b88640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f0f5015f380 0x7f0f500d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57756/0 (socket says 172.21.15.62:57756) 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f55b88640 1 -- 172.21.15.62:0/98132633 learned_addr learned my addr 172.21.15.62:0/98132633 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f56389640 1 --2- 172.21.15.62:0/98132633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f500b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852740870 0 0) 0x7f0f500d3de0 con 0x7f0f5015f380 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f3c003520 con 0x7f0f5015f380 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963710222 0 0) 0x7f0f500d40c0 con 0x7f0f500d22e0 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f500d3de0 con 0x7f0f500d22e0 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1805625252 0 0) 0x7f0f3c003520 con 0x7f0f5015f380 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f56389640 1 -- 172.21.15.62:0/98132633 >> v1:172.21.15.174:6789/0 conn(0x7f0f500d22e0 legacy=0x7f0f500b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f56389640 1 -- 172.21.15.62:0/98132633 >> v1:172.21.15.62:6789/0 conn(0x7f0f5015f380 legacy=0x7f0f500d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.894 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.893+0000 7f0f56389640 1 -- 172.21.15.62:0/98132633 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f500b8280 con 0x7f0f5015f760 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.894+0000 7f0f56389640 1 --2- 172.21.15.62:0/98132633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f500b9190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009dc0 tx=0x7f0f4802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580505ce836dba13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.894+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c001670 con 0x7f0f5015f760 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.894+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4802ae50 con 0x7f0f5015f760 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.894+0000 7f0f54b86640 1 -- 172.21.15.62:0/98132633 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48007760 con 0x7f0f5015f760 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 -- 172.21.15.62:0/98132633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 msgr2=0x7f0f500b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 --2- 172.21.15.62:0/98132633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f500b9190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009dc0 tx=0x7f0f4802a6f0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 -- 172.21.15.62:0/98132633 shutdown_connections 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 --2- 172.21.15.62:0/98132633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f500b9190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 -- 172.21.15.62:0/98132633 >> 172.21.15.62:0/98132633 conn(0x7f0f5009e560 msgr2=0x7f0f500d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 -- 172.21.15.62:0/98132633 shutdown_connections 2024-04-26T22:28:28.896 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.895+0000 7f0f57dec640 1 -- 172.21.15.62:0/98132633 wait complete. 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.896+0000 7f0f57dec640 1 Processor -- start 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.896+0000 7f0f57dec640 1 -- start start 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f5015f380 0x7f0f50160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f50157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0f500d1280 con 0x7f0f500d22e0 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0f500d1100 con 0x7f0f5015f380 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f57dec640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0f500d5b50 con 0x7f0f5015f760 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36378/0 (socket says 172.21.15.62:36378) 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 -- 172.21.15.62:0/520077636 learned_addr learned my addr 172.21.15.62:0/520077636 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55387640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f5015f380 0x7f0f50160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.899 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f56389640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f50157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 -- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 msgr2=0x7f0f50157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f50157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 -- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f5015f380 msgr2=0x7f0f50160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f5015f380 0x7f0f50160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f55b88640 1 -- 172.21.15.62:0/520077636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f50160bf0 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.897+0000 7f0f56389640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f50157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f55b88640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c5d0 tx=0x7f0f4c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c017070 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c005ce0 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f501625b0 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0072b0 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.898+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f50162210 con 0x7f0f500d22e0 2024-04-26T22:28:28.900 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.899+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f500cb880 con 0x7f0f500d22e0 2024-04-26T22:28:28.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.899+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0f4c01a070 con 0x7f0f500d22e0 2024-04-26T22:28:28.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.900+0000 7f0f467fc640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 0x7f0f3406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:28.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.900+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f0f4c08ad50 con 0x7f0f500d22e0 2024-04-26T22:28:28.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.900+0000 7f0f55387640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 0x7f0f3406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:28.901 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.901+0000 7f0f55387640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 0x7f0f3406ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cf080 tx=0x7f0f40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:28.904 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:28.904+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f4c059520 con 0x7f0f500d22e0 2024-04-26T22:28:29.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.090+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f5006a360 con 0x7f0f500d22e0 2024-04-26T22:28:29.091 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.091+0000 7f0f467fc640 1 -- 172.21.15.62:0/520077636 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+13018 (secure 0 0 0) 0x7f0f4c058e20 con 0x7f0f500d22e0 2024-04-26T22:28:29.091 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:29.092 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":34,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:27.770630+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.62:6834/710033080":"2024-04-27T22:28:24.759615+0000","172.21.15.62:6832/710033080":"2024-04-27T22:28:24.759615+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 msgr2=0x7f0f3406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 0x7f0f3406ee40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cf080 tx=0x7f0f40009040 comp rx=0 tx=0).stop 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 msgr2=0x7f0f501542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00c5d0 tx=0x7f0f4c00ca90 comp rx=0 tx=0).stop 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 shutdown_connections 2024-04-26T22:28:29.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f0f3406c980 0x7f0f3406ee40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0f5015f760 0x7f0f50157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f5015f380 0x7f0f50160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 --2- 172.21.15.62:0/520077636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0f500d22e0 0x7f0f501542d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 >> 172.21.15.62:0/520077636 conn(0x7f0f5009e560 msgr2=0x7f0f500b6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 shutdown_connections 2024-04-26T22:28:29.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.094+0000 7f0f57dec640 1 -- 172.21.15.62:0/520077636 wait complete. 2024-04-26T22:28:29.105 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:28:29.105 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:28:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.208+0000 7f9bc5159640 1 Processor -- start 2024-04-26T22:28:29.209 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.208+0000 7f9bc5159640 1 -- start start 2024-04-26T22:28:29.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bc5159640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9bc00d2320 0x7f9bc00d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bc5159640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc0161510 con 0x7f9bc0160800 2024-04-26T22:28:29.211 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bc5159640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc00d0930 con 0x7f9bc01610c0 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bc5159640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc00d0c00 con 0x7f9bc00d2320 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bbed9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f9bc0160800 0x7f9bc00d5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57768/0 (socket says 172.21.15.62:57768) 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.209+0000 7f9bbed9d640 1 -- 172.21.15.62:0/3923071858 learned_addr learned my addr 172.21.15.62:0/3923071858 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbf59e640 1 --2- 172.21.15.62:0/3923071858 >> v2:172.21.15.174:3300/0 conn(0x7f9bc00d2320 0x7f9bc00d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929777375 0 0) 0x7f9bc0161510 con 0x7f9bc0160800 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ba8003520 con 0x7f9bc0160800 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4015241844 0 0) 0x7f9ba8003520 con 0x7f9bc0160800 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 >> v2:172.21.15.174:3300/0 conn(0x7f9bc00d2320 msgr2=0x7f9bc00d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 --2- 172.21.15.62:0/3923071858 >> v2:172.21.15.174:3300/0 conn(0x7f9bc00d2320 0x7f9bc00d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 >> v1:172.21.15.174:6789/0 conn(0x7f9bc01610c0 legacy=0x7f9bc00d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00d0ee0 con 0x7f9bc0160800 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f9bb8001670 con 0x7f9bc0160800 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 >> v1:172.21.15.62:6789/0 conn(0x7f9bc0160800 legacy=0x7f9bc00d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9ba80039b0 0x7f9ba8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.212 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9ba8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba8005120 0x7f9ba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d0c00 con 0x7f9ba80043b0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba800c060 con 0x7f9ba8005120 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba800c260 con 0x7f9ba80039b0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9ba8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbed9d640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9ba80039b0 0x7f9ba8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbf59e640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba8005120 0x7f9ba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 -- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9ba80039b0 msgr2=0x7f9ba8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9ba80039b0 0x7f9ba8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 -- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba8005120 msgr2=0x7f9ba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ba8005120 0x7f9ba800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbe59c640 1 -- 172.21.15.62:0/3923071858 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba8003520 con 0x7f9ba80043b0 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbf59e640 1 -- 172.21.15.62:0/3923071858 reap_dead start 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.210+0000 7f9bbed9d640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9ba80039b0 0x7f9ba8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:29.213 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.211+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9ba8004880 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4001860 tx=0x7f9ba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c685d8c1e1c840 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.211+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4037070 con 0x7f9ba80043b0 2024-04-26T22:28:29.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.211+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb402abe0 con 0x7f9ba80043b0 2024-04-26T22:28:29.214 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.211+0000 7f9bbdd9b640 1 -- 172.21.15.62:0/3923071858 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4001430 con 0x7f9ba80043b0 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 -- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 msgr2=0x7f9ba8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9ba8004880 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4001860 tx=0x7f9ba8005830 comp rx=0 tx=0).stop 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 -- 172.21.15.62:0/3923071858 shutdown_connections 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 --2- 172.21.15.62:0/3923071858 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9ba8004880 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 -- 172.21.15.62:0/3923071858 >> 172.21.15.62:0/3923071858 conn(0x7f9bc009e550 msgr2=0x7f9bc00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 -- 172.21.15.62:0/3923071858 shutdown_connections 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.212+0000 7f9bc5159640 1 -- 172.21.15.62:0/3923071858 wait complete. 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.213+0000 7f9bc5159640 1 Processor -- start 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.213+0000 7f9bc5159640 1 -- start start 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bc015cc10 0x7f9bc00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.217 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bc00b47c0 0x7f9bc00b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d30f0 con 0x7f9ba80043b0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc00d2e10 con 0x7f9bc00b47c0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bc5159640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc00b6300 con 0x7f9bc015cc10 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36386/0 (socket says 172.21.15.62:36386) 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 -- 172.21.15.62:0/506669467 learned_addr learned my addr 172.21.15.62:0/506669467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bc015cc10 0x7f9bc00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbf59e640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bc00b47c0 0x7f9bc00b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 -- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bc015cc10 msgr2=0x7f9bc00cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bc015cc10 0x7f9bc00cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 -- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bc00b47c0 msgr2=0x7f9bc00b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.214+0000 7f9bbed9d640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bc00b47c0 0x7f9bc00b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9bbed9d640 1 -- 172.21.15.62:0/506669467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc00cd2a0 con 0x7f9ba80043b0 2024-04-26T22:28:29.218 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9bbed9d640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8001ba0 tx=0x7f9bb800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800c660 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb8018070 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc00cd580 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800ca00 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.215+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc00cdb40 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.216+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc015cff0 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.216+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f9bb800cba0 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.217+0000 7f9ba77fe640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 0x7f9bac06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.217+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f9bb808b1c0 con 0x7f9ba80043b0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.217+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 0x7f9bac06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.219 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.218+0000 7f9bbe59c640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 0x7f9bac06ee70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4034000 tx=0x7f9bb4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.221 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.221+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9bb80592b0 con 0x7f9ba80043b0 2024-04-26T22:28:29.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.406+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bc006a360 con 0x7f9ba80043b0 2024-04-26T22:28:29.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.407+0000 7f9ba77fe640 1 -- 172.21.15.62:0/506669467 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f9bb805d5b0 con 0x7f9ba80043b0 2024-04-26T22:28:29.407 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 msgr2=0x7f9bac06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 0x7f9bac06ee70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4034000 tx=0x7f9bb4009030 comp rx=0 tx=0).stop 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 msgr2=0x7f9bc015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8001ba0 tx=0x7f9bb800eb60 comp rx=0 tx=0).stop 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 shutdown_connections 2024-04-26T22:28:29.409 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f9bac06c9b0 0x7f9bac06ee70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bc00b47c0 0x7f9bc00b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9bc015cc10 0x7f9bc00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 --2- 172.21.15.62:0/506669467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9ba80043b0 0x7f9bc015d200 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 >> 172.21.15.62:0/506669467 conn(0x7f9bc009e550 msgr2=0x7f9bc00b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 shutdown_connections 2024-04-26T22:28:29.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.409+0000 7f9bc5159640 1 -- 172.21.15.62:0/506669467 wait complete. 2024-04-26T22:28:29.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2024-04-26T22:28:29.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.521+0000 7fec32693640 1 Processor -- start 2024-04-26T22:28:29.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.521+0000 7fec32693640 1 -- start start 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec32693640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec32693640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec2c15f760 0x7fec2c0b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec32693640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec2c0d3de0 con 0x7fec2c15f380 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec32693640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fec2c0d43a0 con 0x7fec2c15f760 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec32693640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0d40c0 con 0x7fec2c0d22e0 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec30c30640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fec2c15f760 0x7fec2c0b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec2bfff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fec2c15f380 0x7fec2c0d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57776/0 (socket says 172.21.15.62:57776) 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec2bfff640 1 -- 172.21.15.62:0/546410 learned_addr learned my addr 172.21.15.62:0/546410 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.522+0000 7fec2b7fe640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec30c30640 1 -- 172.21.15.62:0/546410 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 msgr2=0x7fec2c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec30c30640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec30c30640 1 -- 172.21.15.62:0/546410 >> v1:172.21.15.62:6789/0 conn(0x7fec2c15f380 legacy=0x7fec2c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec30c30640 1 -- 172.21.15.62:0/546410 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0b8280 con 0x7fec2c15f760 2024-04-26T22:28:29.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec2affd640 1 -- 172.21.15.62:0/546410 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859951990 0 0) 0x7fec2c0d3de0 con 0x7fec2c15f380 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec30c30640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f760 0x7fec2c0b8fc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec24009d90 tx=0x7fec2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9806028d850d8ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec2affd640 1 -- 172.21.15.62:0/546410 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24007740 con 0x7fec2c15f760 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec2affd640 1 -- 172.21.15.62:0/546410 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec240078e0 con 0x7fec2c15f760 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.523+0000 7fec2affd640 1 -- 172.21.15.62:0/546410 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24007cb0 con 0x7fec2c15f760 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 -- 172.21.15.62:0/546410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f760 msgr2=0x7fec2c0b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f760 0x7fec2c0b8fc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fec24009d90 tx=0x7fec2402a6f0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 -- 172.21.15.62:0/546410 shutdown_connections 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f760 0x7fec2c0b8fc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 --2- 172.21.15.62:0/546410 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 -- 172.21.15.62:0/546410 >> 172.21.15.62:0/546410 conn(0x7fec2c09e560 msgr2=0x7fec2c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 -- 172.21.15.62:0/546410 shutdown_connections 2024-04-26T22:28:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.524+0000 7fec32693640 1 -- 172.21.15.62:0/546410 wait complete. 2024-04-26T22:28:29.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.525+0000 7fec32693640 1 Processor -- start 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.525+0000 7fec32693640 1 -- start start 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec2c15f760 0x7fec2c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d1280 con 0x7fec2c15f380 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d1100 con 0x7fec2c15f760 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec32693640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0d5b50 con 0x7fec2c0d22e0 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec2b7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36412/0 (socket says 172.21.15.62:36412) 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec2b7fe640 1 -- 172.21.15.62:0/1081585922 learned_addr learned my addr 172.21.15.62:0/1081585922 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec2bfff640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.526+0000 7fec30c30640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec2c15f760 0x7fec2c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 -- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 msgr2=0x7fec2c0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 -- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec2c15f760 msgr2=0x7fec2c152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec2c15f760 0x7fec2c152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 -- 172.21.15.62:0/1081585922 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0cb970 con 0x7fec2c15f380 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec2b7fe640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00d800 tx=0x7fec1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00b6c0 con 0x7fec2c15f380 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c019070 con 0x7fec2c15f380 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c0cbed0 con 0x7fec2c15f380 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00ba80 con 0x7fec2c15f380 2024-04-26T22:28:29.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.527+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c161330 con 0x7fec2c15f380 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.528+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c06a360 con 0x7fec2c15f380 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.528+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fec1c00bc20 con 0x7fec2c15f380 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.529+0000 7fec28ff9640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 0x7fec1806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.529+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fec1c08bc30 con 0x7fec2c15f380 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.529+0000 7fec2bfff640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 0x7fec1806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:29.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.529+0000 7fec2bfff640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 0x7fec1806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec20000f60 tx=0x7fec20007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:29.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.533+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec2c0cb970 con 0x7fec2c15f380 2024-04-26T22:28:29.796 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:29.796+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) -- 0x7fec2c0b90e0 con 0x7fec2c15f380 2024-04-26T22:28:30.782 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.781+0000 7fec28ff9640 1 -- 172.21.15.62:0/1081585922 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v93) ==== 175+0+0 (secure 0 0 0) 0x7fec1c060030 con 0x7fec2c15f380 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 msgr2=0x7fec1806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 0x7fec1806ee40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec20000f60 tx=0x7fec20007d50 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 msgr2=0x7fec2c0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00d800 tx=0x7fec1c00dcc0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 shutdown_connections 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fec1806c980 0x7fec1806ee40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec2c15f760 0x7fec2c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fec2c15f380 0x7fec2c0ca090 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 --2- 172.21.15.62:0/1081585922 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec2c0d22e0 0x7fec2c0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 >> 172.21.15.62:0/1081585922 conn(0x7fec2c09e560 msgr2=0x7fec2c0b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 shutdown_connections 2024-04-26T22:28:30.785 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.784+0000 7fec32693640 1 -- 172.21.15.62:0/1081585922 wait complete. 2024-04-26T22:28:30.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T22:28:30.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.908+0000 7f7a03b7f640 1 Processor -- start 2024-04-26T22:28:30.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.908+0000 7f7a03b7f640 1 -- start start 2024-04-26T22:28:30.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a03b7f640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79fc0cfba0 0x7f79fc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a03b7f640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc0cdb90 con 0x7f79fc15d830 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a03b7f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc0d1ff0 con 0x7f79fc0b25e0 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a03b7f640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f79fc0b2390 con 0x7f79fc0cfba0 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a0191b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f79fc0cfba0 0x7f79fc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a0211c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f79fc15d830 0x7f79fc0ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:57780/0 (socket says 172.21.15.62:57780) 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.909+0000 7f7a0211c640 1 -- 172.21.15.62:0/982062612 learned_addr learned my addr 172.21.15.62:0/982062612 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a0191b640 1 -- 172.21.15.62:0/982062612 >> v1:172.21.15.174:6789/0 conn(0x7f79fc0b25e0 legacy=0x7f79fc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a0191b640 1 -- 172.21.15.62:0/982062612 >> v1:172.21.15.62:6789/0 conn(0x7f79fc15d830 legacy=0x7f79fc0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a00919640 1 -- 172.21.15.62:0/982062612 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545417126 0 0) 0x7f79fc0cdb90 con 0x7f79fc15d830 2024-04-26T22:28:30.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a0191b640 1 -- 172.21.15.62:0/982062612 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc0d1a30 con 0x7f79fc0cfba0 2024-04-26T22:28:30.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a0191b640 1 --2- 172.21.15.62:0/982062612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc0cfba0 0x7f79fc0b4320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd22b5f1ef37eef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:30.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.910+0000 7f7a00919640 1 -- 172.21.15.62:0/982062612 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8037070 con 0x7f79fc0cfba0 2024-04-26T22:28:30.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a00919640 1 -- 172.21.15.62:0/982062612 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f8032440 con 0x7f79fc0cfba0 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a00919640 1 -- 172.21.15.62:0/982062612 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8032810 con 0x7f79fc0cfba0 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 -- 172.21.15.62:0/982062612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc0cfba0 msgr2=0x7f79fc0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/982062612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc0cfba0 0x7f79fc0b4320 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802a1a0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 -- 172.21.15.62:0/982062612 shutdown_connections 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/982062612 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc0cfba0 0x7f79fc0b4320 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 -- 172.21.15.62:0/982062612 >> 172.21.15.62:0/982062612 conn(0x7f79fc0d2f80 msgr2=0x7f79fc0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 -- 172.21.15.62:0/982062612 shutdown_connections 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.911+0000 7f7a03b7f640 1 -- 172.21.15.62:0/982062612 wait complete. 2024-04-26T22:28:30.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.912+0000 7f7a03b7f640 1 Processor -- start 2024-04-26T22:28:30.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.912+0000 7f7a03b7f640 1 -- start start 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79fc0b25e0 0x7f79fc0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79fc0cfba0 0x7f79fc15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0cff80 con 0x7f79fc15d830 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc0cde30 con 0x7f79fc0b25e0 2024-04-26T22:28:30.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a03b7f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0d0100 con 0x7f79fc0cfba0 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a0211c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a0211c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36434/0 (socket says 172.21.15.62:36434) 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a0211c640 1 -- 172.21.15.62:0/2215447993 learned_addr learned my addr 172.21.15.62:0/2215447993 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a0191b640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79fc0b25e0 0x7f79fc0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.913+0000 7f7a0111a640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79fc0cfba0 0x7f79fc15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 -- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79fc0cfba0 msgr2=0x7f79fc15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79fc0cfba0 0x7f79fc15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 -- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79fc0b25e0 msgr2=0x7f79fc0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79fc0b25e0 0x7f79fc0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 -- 172.21.15.62:0/2215447993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc09ea80 con 0x7f79fc15d830 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a0211c640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c700 tx=0x7f79f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4018070 con 0x7f79fc15d830 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f400ce10 con 0x7f79fc15d830 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc09ed20 con 0x7f79fc15d830 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f400a860 con 0x7f79fc15d830 2024-04-26T22:28:30.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.914+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc15acf0 con 0x7f79fc15d830 2024-04-26T22:28:30.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.915+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc06a360 con 0x7f79fc15d830 2024-04-26T22:28:30.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.916+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f79f401b070 con 0x7f79fc15d830 2024-04-26T22:28:30.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.916+0000 7f79ee7fc640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 0x7f79e006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:30.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.916+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7f79f408ae00 con 0x7f79fc15d830 2024-04-26T22:28:30.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.916+0000 7f7a0191b640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 0x7f79e006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:30.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.917+0000 7f7a0191b640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 0x7f79e006ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:30.920 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:30.920+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f79f4059540 con 0x7f79fc15d830 2024-04-26T22:28:31.109 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.109+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f79fc0d06c0 con 0x7f79fc15d830 2024-04-26T22:28:31.112 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.112+0000 7f79ee7fc640 1 -- 172.21.15.62:0/2215447993 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f79f4058e40 con 0x7f79fc15d830 2024-04-26T22:28:31.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 msgr2=0x7f79e006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.114 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 0x7f79e006ee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f79f8034000 tx=0x7f79f802e000 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 msgr2=0x7f79fc09f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c700 tx=0x7f79f400cbc0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 shutdown_connections 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f79e006c980 0x7f79e006ee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f79fc15d830 0x7f79fc09f800 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79fc0cfba0 0x7f79fc15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 --2- 172.21.15.62:0/2215447993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79fc0b25e0 0x7f79fc0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 >> 172.21.15.62:0/2215447993 conn(0x7f79fc0d2f80 msgr2=0x7f79fc0ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 shutdown_connections 2024-04-26T22:28:31.115 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.114+0000 7f7a03b7f640 1 -- 172.21.15.62:0/2215447993 wait complete. 2024-04-26T22:28:31.125 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T22:28:31.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:28:31.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T22:28:31.212 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T22:28:31.212 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:31.212 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:31.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056113,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T22:28:31.228 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:31.228 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:31.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17182 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T22:28:31.246 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:31.246 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:31.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056113,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19663 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T22:28:31.283 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:31.283 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:31.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056096,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17182 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T22:28:31.303 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T22:28:31.304 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:31.423 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.423+0000 7fefc1159640 1 Processor -- start 2024-04-26T22:28:31.424 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.423+0000 7fefc1159640 1 -- start start 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefc1159640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fefbc15cf70 0x7fefbc06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefc1159640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 0x7fefbc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefc1159640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefbc070210 con 0x7fefbc163b70 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefc1159640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fefbc163090 con 0x7fefbc15cf70 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefc1159640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc163370 con 0x7fefbc06fbf0 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefbb59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 0x7fefbc06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.424+0000 7fefbad9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fefbc15cf70 0x7fefbc06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefba59c640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fefbc163b70 0x7fefbc06f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:38930/0 (socket says 172.21.15.62:38930) 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefba59c640 1 -- 172.21.15.62:0/1859128287 learned_addr learned my addr 172.21.15.62:0/1859128287 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:31.425 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefbad9d640 1 -- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 msgr2=0x7fefbc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefbad9d640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 0x7fefbc06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefbad9d640 1 -- 172.21.15.62:0/1859128287 >> v1:172.21.15.174:6789/0 conn(0x7fefbc163b70 legacy=0x7fefbc06f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefbad9d640 1 -- 172.21.15.62:0/1859128287 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc163650 con 0x7fefbc15cf70 2024-04-26T22:28:31.426 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.425+0000 7fefbb59e640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 0x7fefbc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefbad9d640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc15cf70 0x7fefbc06c240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5edcb50e5c73e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefb9d9b640 1 -- 172.21.15.62:0/1859128287 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037070 con 0x7fefbc15cf70 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefb9d9b640 1 -- 172.21.15.62:0/1859128287 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb4032440 con 0x7fefbc15cf70 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefb9d9b640 1 -- 172.21.15.62:0/1859128287 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4032810 con 0x7fefbc15cf70 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 -- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc15cf70 msgr2=0x7fefbc06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc15cf70 0x7fefbc06c240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefb40097a0 tx=0x7fefb402a180 comp rx=0 tx=0).stop 2024-04-26T22:28:31.427 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 -- 172.21.15.62:0/1859128287 shutdown_connections 2024-04-26T22:28:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc06fbf0 0x7fefbc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 --2- 172.21.15.62:0/1859128287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc15cf70 0x7fefbc06c240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 -- 172.21.15.62:0/1859128287 >> 172.21.15.62:0/1859128287 conn(0x7fefbc0b3b00 msgr2=0x7fefbc0b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 -- 172.21.15.62:0/1859128287 shutdown_connections 2024-04-26T22:28:31.428 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.426+0000 7fefc1159640 1 -- 172.21.15.62:0/1859128287 wait complete. 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.428+0000 7fefc1159640 1 Processor -- start 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.428+0000 7fefc1159640 1 -- start start 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 0x7fefbc14a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 0x7fefbc14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc15fbc0 con 0x7fefbc06fbf0 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc15fa40 con 0x7fefbc163b70 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fefbc15fd40 con 0x7fefbc15cf70 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36436/0 (socket says 172.21.15.62:36436) 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 -- 172.21.15.62:0/1550613536 learned_addr learned my addr 172.21.15.62:0/1550613536 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:31.430 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefba59c640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 0x7fefbc14a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbb59e640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 0x7fefbc14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 -- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 msgr2=0x7fefbc14a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 0x7fefbc14a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 -- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 msgr2=0x7fefbc14a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 0x7fefbc14a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 -- 172.21.15.62:0/1550613536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc15e960 con 0x7fefbc06fbf0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefba59c640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 0x7fefbc14a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbb59e640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 0x7fefbc14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefbad9d640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefb4035040 tx=0x7fefb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4037040 con 0x7fefbc06fbf0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb4030500 con 0x7fefbc06fbf0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40308d0 con 0x7fefbc06fbf0 2024-04-26T22:28:31.431 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc15ec40 con 0x7fefbc06fbf0 2024-04-26T22:28:31.432 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.429+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc15f200 con 0x7fefbc06fbf0 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.430+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc0693e0 con 0x7fefbc06fbf0 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.431+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fefb4044020 con 0x7fefbc06fbf0 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.431+0000 7fefa37fe640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 0x7fefa806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.431+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fefb40abb00 con 0x7fefbc06fbf0 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.431+0000 7fefba59c640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 0x7fefa806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.433 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.432+0000 7fefba59c640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 0x7fefa806ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0af930 tx=0x7fefb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.435 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.435+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefb407a240 con 0x7fefbc06fbf0 2024-04-26T22:28:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.658+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefbc158660 con 0x7fefbc06fbf0 2024-04-26T22:28:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.659+0000 7fefa37fe640 1 -- 172.21.15.62:0/1550613536 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3872 (secure 0 0 0) 0x7fefb4079b40 con 0x7fefbc06fbf0 2024-04-26T22:28:31.659 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:31.659 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4712,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6836/2856482907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2856482907},{"type":"v1","addr":"172.21.15.174:6836","nonce":2856482907}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1678359264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1678359264},{"type":"v1","addr":"172.21.15.174:6837","nonce":1678359264}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4721,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/1370121505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":1370121505},{"type":"v1","addr":"172.21.15.62:6835","nonce":1370121505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5466,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/737890540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":737890540},{"type":"v1","addr":"172.21.15.62:6834","nonce":737890540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-26T22:28:31.659 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 19 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 msgr2=0x7fefa806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 0x7fefa806ee40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0af930 tx=0x7fefb0009040 comp rx=0 tx=0).stop 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 msgr2=0x7fefbc0b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefb4035040 tx=0x7fefb40046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 shutdown_connections 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fefa806c980 0x7fefa806ee40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefbc163b70 0x7fefbc14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fefbc15cf70 0x7fefbc14a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 --2- 172.21.15.62:0/1550613536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefbc06fbf0 0x7fefbc0b2dd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 >> 172.21.15.62:0/1550613536 conn(0x7fefbc0b3b00 msgr2=0x7fefbc0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:31.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 shutdown_connections 2024-04-26T22:28:31.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.661+0000 7fefc1159640 1 -- 172.21.15.62:0/1550613536 wait complete. 2024-04-26T22:28:31.672 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T22:28:31.672 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T22:28:31.672 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T22:28:31.672 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:28:31.673 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T22:28:31.673 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T22:28:31.673 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T22:28:31.673 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:28:31.673 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T22:28:31.673 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T22:28:31.674 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T22:28:31.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:28:31.674 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T22:28:31.674 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T22:28:31.674 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T22:28:31.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T22:28:31.683 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T22:28:31.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T22:28:31.684 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T22:28:31.684 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T22:28:31.686 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T22:28:31.686 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T22:28:31.687 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T22:28:31.687 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T22:28:31.725 INFO:tasks.ceph.mds.a:Started 2024-04-26T22:28:31.726 DEBUG:teuthology.parallel:result is None 2024-04-26T22:28:31.726 INFO:tasks.ceph.mds.c:Started 2024-04-26T22:28:31.726 DEBUG:teuthology.parallel:result is None 2024-04-26T22:28:31.728 INFO:tasks.ceph.mds.b:Started 2024-04-26T22:28:31.729 DEBUG:teuthology.parallel:result is None 2024-04-26T22:28:31.729 INFO:tasks.ceph.mds.d:Started 2024-04-26T22:28:31.729 DEBUG:teuthology.parallel:result is None 2024-04-26T22:28:31.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T22:28:31.801 INFO:tasks.ceph.mds.a.smithi062.stdout:starting mds.a at 2024-04-26T22:28:31.802 INFO:tasks.ceph.mds.c.smithi062.stdout:starting mds.c at 2024-04-26T22:28:31.812 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2024-04-26T22:28:31.813 INFO:tasks.ceph.mds.d.smithi174.stdout:starting mds.d at 2024-04-26T22:28:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.836+0000 7fd67bd24640 1 Processor -- start 2024-04-26T22:28:31.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.837+0000 7fd67bd24640 1 -- start start 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.837+0000 7fd67bd24640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd674162b40 0x7fd6740d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.837+0000 7fd67bd24640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 0x7fd6740d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd67bd24640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6740d0210 con 0x7fd6740d1850 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd67bd24640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd6740d41c0 con 0x7fd674162b40 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd67bd24640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740d04f0 con 0x7fd6740d3820 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd674162b40 0x7fd6740d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.838 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd674162b40 0x7fd6740d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36480/0 (socket says 172.21.15.62:36480) 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 -- 172.21.15.62:0/1355020364 learned_addr learned my addr 172.21.15.62:0/1355020364 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd6792bf640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 0x7fd6740d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 -- 172.21.15.62:0/1355020364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 msgr2=0x7fd6740d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 0x7fd6740d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 -- 172.21.15.62:0/1355020364 >> v1:172.21.15.174:6789/0 conn(0x7fd6740d1850 legacy=0x7fd6740d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd679ac0640 1 -- 172.21.15.62:0/1355020364 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6740d07d0 con 0x7fd674162b40 2024-04-26T22:28:31.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.838+0000 7fd6792bf640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 0x7fd6740d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd679ac0640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd674162b40 0x7fd6740d5bd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd664009870 tx=0x7fd66402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f302f41d41c9746f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd678abe640 1 -- 172.21.15.62:0/1355020364 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664037070 con 0x7fd674162b40 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd678abe640 1 -- 172.21.15.62:0/1355020364 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66402ae50 con 0x7fd674162b40 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd678abe640 1 -- 172.21.15.62:0/1355020364 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664032860 con 0x7fd674162b40 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 -- 172.21.15.62:0/1355020364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd674162b40 msgr2=0x7fd6740d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd674162b40 0x7fd6740d5bd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd664009870 tx=0x7fd66402a300 comp rx=0 tx=0).stop 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 -- 172.21.15.62:0/1355020364 shutdown_connections 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d3820 0x7fd6740d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 --2- 172.21.15.62:0/1355020364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd674162b40 0x7fd6740d5bd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 -- 172.21.15.62:0/1355020364 >> 172.21.15.62:0/1355020364 conn(0x7fd67409e550 msgr2=0x7fd6740d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 -- 172.21.15.62:0/1355020364 shutdown_connections 2024-04-26T22:28:31.841 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.839+0000 7fd67bd24640 1 -- 172.21.15.62:0/1355020364 wait complete. 2024-04-26T22:28:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 Processor -- start 2024-04-26T22:28:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 -- start start 2024-04-26T22:28:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d1850 0x7fd6741f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.843 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674162b40 0x7fd6741f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740b5bb0 con 0x7fd6740d3820 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6740d0d90 con 0x7fd674162b40 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd67bd24640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd6740d6110 con 0x7fd6740d1850 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.841+0000 7fd6792bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:36492/0 (socket says 172.21.15.62:36492) 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 -- 172.21.15.62:0/354330467 learned_addr learned my addr 172.21.15.62:0/354330467 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd67a2c1640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674162b40 0x7fd6741f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd679ac0640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d1850 0x7fd6741f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 -- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d1850 msgr2=0x7fd6741f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d1850 0x7fd6741f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 -- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674162b40 msgr2=0x7fd6741f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674162b40 0x7fd6741f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 -- 172.21.15.62:0/354330467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6741f6110 con 0x7fd6740d3820 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd6792bf640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d7d0 tx=0x7fd67000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670004030 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6700041d0 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700096a0 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6741f92d0 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.842+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6740b9680 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.843+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67406a360 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.844+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd670009840 con 0x7fd6740d3820 2024-04-26T22:28:31.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.844+0000 7fd66a7fc640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 0x7fd65806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:31.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.844+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4943+0+0 (secure 0 0 0) 0x7fd67008ba70 con 0x7fd6740d3820 2024-04-26T22:28:31.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.844+0000 7fd679ac0640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 0x7fd65806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:31.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.845+0000 7fd679ac0640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 0x7fd65806ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd664034000 tx=0x7fd66402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:31.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:31.848+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd67005a240 con 0x7fd6740d3820 2024-04-26T22:28:32.049 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:32.048+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd674075fb0 con 0x7fd6740d3820 2024-04-26T22:28:33.127 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.127+0000 7fd66a7fc640 1 -- 172.21.15.62:0/354330467 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7fd670059b40 con 0x7fd6740d3820 2024-04-26T22:28:33.128 INFO:teuthology.orchestra.run.smithi062.stderr: removed all blocklist entries 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 msgr2=0x7fd65806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 0x7fd65806ee40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd664034000 tx=0x7fd66402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 msgr2=0x7fd6741f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd67000d7d0 tx=0x7fd67000dc90 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 shutdown_connections 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd65806c980 0x7fd65806ee40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674162b40 0x7fd6741f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6740d3820 0x7fd6741f89e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 --2- 172.21.15.62:0/354330467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd6740d1850 0x7fd6741f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 >> 172.21.15.62:0/354330467 conn(0x7fd67409e550 msgr2=0x7fd67406d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 shutdown_connections 2024-04-26T22:28:33.131 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.130+0000 7fd67bd24640 1 -- 172.21.15.62:0/354330467 wait complete. 2024-04-26T22:28:33.143 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T22:28:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 Processor -- start 2024-04-26T22:28:33.246 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 -- start start 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f259c0d3c80 0x7f259c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259c0d0210 con 0x7f259c162b40 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259c0d04f0 con 0x7f259c0d3820 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.246+0000 7f25a1fd4640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f259c0d07d0 con 0x7f259c0d3c80 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f259c0d3c80 0x7f259c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f259c0d3c80 0x7f259c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60460/0 (socket says 172.21.15.62:60460) 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259b7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f259c162b40 0x7f259c0d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56572/0 (socket says 172.21.15.62:56572) 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 -- 172.21.15.62:0/892838263 learned_addr learned my addr 172.21.15.62:0/892838263 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 -- 172.21.15.62:0/892838263 >> v1:172.21.15.174:6789/0 conn(0x7f259c0d3820 legacy=0x7f259c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 -- 172.21.15.62:0/892838263 >> v1:172.21.15.62:6789/0 conn(0x7f259c162b40 legacy=0x7f259c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.247+0000 7f259bfff640 1 -- 172.21.15.62:0/892838263 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c0d0ab0 con 0x7f259c0d3c80 2024-04-26T22:28:33.248 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f259bfff640 1 --2- 172.21.15.62:0/892838263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0b52b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2594009e50 tx=0x7f259402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce692b2afb80dfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f259a7fc640 1 -- 172.21.15.62:0/892838263 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594036070 con 0x7f259c0d3c80 2024-04-26T22:28:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f259a7fc640 1 -- 172.21.15.62:0/892838263 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2594031600 con 0x7f259c0d3c80 2024-04-26T22:28:33.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f259a7fc640 1 -- 172.21.15.62:0/892838263 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25940319f0 con 0x7f259c0d3c80 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 -- 172.21.15.62:0/892838263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 msgr2=0x7f259c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/892838263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0b52b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2594009e50 tx=0x7f259402a7c0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 -- 172.21.15.62:0/892838263 shutdown_connections 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/892838263 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0b52b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 -- 172.21.15.62:0/892838263 >> 172.21.15.62:0/892838263 conn(0x7f259c09e550 msgr2=0x7f259c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 -- 172.21.15.62:0/892838263 shutdown_connections 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.248+0000 7f25a1fd4640 1 -- 172.21.15.62:0/892838263 wait complete. 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.249+0000 7f25a1fd4640 1 Processor -- start 2024-04-26T22:28:33.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 -- start start 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f259c0d3820 0x7f259c152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f259c162b40 0x7f259c0b9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f259c0d12b0 con 0x7f259c0d3c80 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f259c0d4110 con 0x7f259c162b40 2024-04-26T22:28:33.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f25a1fd4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f259c0cfe30 con 0x7f259c0d3820 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f259affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.250+0000 7f259affd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60468/0 (socket says 172.21.15.62:60468) 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 -- 172.21.15.62:0/2250753949 learned_addr learned my addr 172.21.15.62:0/2250753949 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259bfff640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f259c162b40 0x7f259c0b9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259b7fe640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f259c0d3820 0x7f259c152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 -- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f259c0d3820 msgr2=0x7f259c152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f259c0d3820 0x7f259c152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 -- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f259c162b40 msgr2=0x7f259c0b9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f259c162b40 0x7f259c0b9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 -- 172.21.15.62:0/2250753949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c152bd0 con 0x7f259c0d3c80 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f259affd640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f259000c3c0 tx=0x7f259000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25900052d0 con 0x7f259c0d3c80 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2590005490 con 0x7f259c0d3c80 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590007260 con 0x7f259c0d3c80 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c0ba1e0 con 0x7f259c0d3c80 2024-04-26T22:28:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.251+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c0ba7a0 con 0x7f259c0d3c80 2024-04-26T22:28:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.252+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c06a360 con 0x7f259c0d3c80 2024-04-26T22:28:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.253+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2590007400 con 0x7f259c0d3c80 2024-04-26T22:28:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.253+0000 7f2583fff640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 0x7f258806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.253+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f259008ab50 con 0x7f259c0d3c80 2024-04-26T22:28:33.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.253+0000 7f259b7fe640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 0x7f258806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.254+0000 7f259b7fe640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 0x7f258806ee70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.257+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2590058d20 con 0x7f259c0d3c80 2024-04-26T22:28:33.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.521+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f259c0d2960 con 0x7f259c0d3c80 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.522+0000 7f2583fff640 1 -- 172.21.15.62:0/2250753949 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f259005f020 con 0x7f259c0d3c80 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout: "auth_dump": [ 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.a", 2024-04-26T22:28:33.522 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmWg+9NRAAdnpMVpgAaGZ21j6W0Tbs+w==", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.b", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmwQbLBRAAuP30XZDLsJaBm6BNHTqFCA==", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:28:33.523 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.c", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm/wIqARAA55sdB593n/Skd4uA1syuSg==", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.524 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mds.d", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm0jNbEBAAudci4vHH/MXyeG7tCWL1Nw==", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.0", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQALKixmMfTWLxAAg/vI2T8zb3xX7r3ZgdrTFg==", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.525 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.1", 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAOKixmfyK2LhAAHbPw2zCa8HuUWqsXiFPkPQ==", 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.526 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.2", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQARKixmSo+dLBAAiPoVs57GokukpcqH47rfAw==", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.3", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAUKixmXiJYKxAAo5iZwxDjIP6rn0cRysJxfg==", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.527 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.4", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAaKixmVSbbMBAA4oChxHAndPdjzYm3K59Acg==", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.5", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAdKixmTA5LLxAAtirJ58cd12oGcGYZ4W4eQg==", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.528 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.6", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAgKixm3+KjLRAAjf0ELmtEHU5hdTojxCPJDw==", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "osd.7", 2024-04-26T22:28:33.529 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAjKixmm0jnKxAA7IYEt06xhis1drk5h0zH2w==", 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow profile osd", 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile osd", 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.0", 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ==", 2024-04-26T22:28:33.530 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow r", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.1", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixm20S/HRAAlRqwTgIuwG+sjvnjS87R1g==", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.531 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.2", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmZOyjGhAAxeZlr2qggbHBj9ZjmCsOqA==", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.3", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAIKixmfI08IxAAbwPyTVFRY5J5Vw08V31Ocw==", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow r", 2024-04-26T22:28:33.532 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow rw", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow rwx" 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "client.admin", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmlpqGGhAAVIn6IHaa2aYjviVKuQEqFQ==", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "mgr": "allow *", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow *", 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.533 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.x", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixmgseXMhAAtwa5QiDplGCQk5DQlFHHvg==", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "entity": "mgr.y", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "key": "AQAHKixm0y23LhAA5KnsIydYZchjnmyUwLWl1w==", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": { 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": "allow *", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "mon": "allow profile mgr", 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: "osd": "allow *" 2024-04-26T22:28:33.534 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.535 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:33.535 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-26T22:28:33.535 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-26T22:28:33.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 msgr2=0x7f258806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 0x7f258806ee70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f258c000f60 tx=0x7f258c000ed0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 msgr2=0x7f259c0ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f259000c3c0 tx=0x7f259000c880 comp rx=0 tx=0).stop 2024-04-26T22:28:33.536 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 shutdown_connections 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f258806c9b0 0x7f258806ee70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f259c162b40 0x7f259c0b9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f259c0d3c80 0x7f259c0ce980 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 --2- 172.21.15.62:0/2250753949 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f259c0d3820 0x7f259c152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 >> 172.21.15.62:0/2250753949 conn(0x7f259c09e550 msgr2=0x7f259c06d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 shutdown_connections 2024-04-26T22:28:33.537 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.524+0000 7f25a1fd4640 1 -- 172.21.15.62:0/2250753949 wait complete. 2024-04-26T22:28:33.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.638+0000 7fb84cb22640 1 Processor -- start 2024-04-26T22:28:33.638 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.638+0000 7fb84cb22640 1 -- start start 2024-04-26T22:28:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84cb22640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 0x7fb8480d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84cb22640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb8480cff40 0x7fb8480ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84cb22640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8480b1d70 con 0x7fb8480b2050 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84cb22640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb848163220 con 0x7fb8480cff40 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84cb22640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480d0eb0 con 0x7fb848162e40 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb846d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb8480cff40 0x7fb8480ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb84659c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 0x7fb8480d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb846d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb8480cff40 0x7fb8480ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.62:51074/0 (socket says 172.21.15.62:51074) 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb83ffff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb8480b2050 0x7fb8480cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:49410/0 (socket says 172.21.15.62:49410) 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.639+0000 7fb846d9d640 1 -- 172.21.15.62:0/112240594 learned_addr learned my addr 172.21.15.62:0/112240594 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb846d9d640 1 -- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 msgr2=0x7fb8480d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb846d9d640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 0x7fb8480d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb846d9d640 1 -- 172.21.15.62:0/112240594 >> v1:172.21.15.174:6789/0 conn(0x7fb8480b2050 legacy=0x7fb8480cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb846d9d640 1 -- 172.21.15.62:0/112240594 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8480d1960 con 0x7fb8480cff40 2024-04-26T22:28:33.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb84659c640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 0x7fb8480d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb846d9d640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8480cff40 0x7fb8480ce010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb840009e50 tx=0x7fb84002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=494444e79cffb32b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.640+0000 7fb845d9b640 1 -- 172.21.15.62:0/112240594 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840036070 con 0x7fb8480cff40 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb845d9b640 1 -- 172.21.15.62:0/112240594 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb840031600 con 0x7fb8480cff40 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb845d9b640 1 -- 172.21.15.62:0/112240594 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8400319d0 con 0x7fb8480cff40 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 -- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8480cff40 msgr2=0x7fb8480ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8480cff40 0x7fb8480ce010 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb840009e50 tx=0x7fb84002a680 comp rx=0 tx=0).stop 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 -- 172.21.15.62:0/112240594 shutdown_connections 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8480cff40 0x7fb8480ce010 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 --2- 172.21.15.62:0/112240594 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb848162e40 0x7fb8480d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 -- 172.21.15.62:0/112240594 >> 172.21.15.62:0/112240594 conn(0x7fb84809ba50 msgr2=0x7fb84809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 -- 172.21.15.62:0/112240594 shutdown_connections 2024-04-26T22:28:33.643 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.641+0000 7fb84cb22640 1 -- 172.21.15.62:0/112240594 wait complete. 2024-04-26T22:28:33.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.642+0000 7fb84cb22640 1 Processor -- start 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.642+0000 7fb84cb22640 1 -- start start 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8480cff40 0x7fb848159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb848162e40 0x7fb8480b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b24b0 con 0x7fb8480b2050 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480d1f20 con 0x7fb848162e40 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84cb22640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480b2630 con 0x7fb8480cff40 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60470/0 (socket says 172.21.15.62:60470) 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb84659c640 1 -- 172.21.15.62:0/1572257770 learned_addr learned my addr 172.21.15.62:0/1572257770 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.643+0000 7fb83ffff640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8480cff40 0x7fb848159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb846d9d640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb848162e40 0x7fb8480b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 -- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8480cff40 msgr2=0x7fb848159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8480cff40 0x7fb848159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.645 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 -- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb848162e40 msgr2=0x7fb8480b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb848162e40 0x7fb8480b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 -- 172.21.15.62:0/1572257770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84815bc50 con 0x7fb8480b2050 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84659c640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e990 tx=0x7fb83800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838017070 con 0x7fb8480b2050 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83800c9e0 con 0x7fb8480b2050 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cdb0 con 0x7fb8480b2050 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8480b7290 con 0x7fb8480b2050 2024-04-26T22:28:33.646 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.644+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8480b7b30 con 0x7fb8480b2050 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.645+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8480b7680 con 0x7fb8480b2050 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.646+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb83801a070 con 0x7fb8480b2050 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.646+0000 7fb83effd640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 0x7fb82806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.646+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7fb83808b740 con 0x7fb8480b2050 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.646+0000 7fb83ffff640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 0x7fb82806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.647 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.647+0000 7fb83ffff640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 0x7fb82806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8480d2650 tx=0x7fb830007ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.651+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb838059ed0 con 0x7fb8480b2050 2024-04-26T22:28:33.836 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.836+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb84806a360 con 0x7fb8480b2050 2024-04-26T22:28:33.837 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.837+0000 7fb83effd640 1 -- 172.21.15.62:0/1572257770 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+12891 (secure 0 0 0) 0x7fb8380597d0 con 0x7fb8480b2050 2024-04-26T22:28:33.837 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:33.837 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":39,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:33.126668+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 msgr2=0x7fb82806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 0x7fb82806ee40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb8480d2650 tx=0x7fb830007ce0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 msgr2=0x7fb84815b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e990 tx=0x7fb83800ee50 comp rx=0 tx=0).stop 2024-04-26T22:28:33.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 shutdown_connections 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb82806c980 0x7fb82806ee40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb848162e40 0x7fb8480b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb8480cff40 0x7fb848159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 --2- 172.21.15.62:0/1572257770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8480b2050 0x7fb84815b710 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 >> 172.21.15.62:0/1572257770 conn(0x7fb84809ba50 msgr2=0x7fb84806df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 shutdown_connections 2024-04-26T22:28:33.840 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.839+0000 7fb84cb22640 1 -- 172.21.15.62:0/1572257770 wait complete. 2024-04-26T22:28:33.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T22:27:01.653574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T22:28:33.851 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T22:28:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.954+0000 7f4267ea9640 1 Processor -- start 2024-04-26T22:28:33.954 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.954+0000 7f4267ea9640 1 -- start start 2024-04-26T22:28:33.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4267ea9640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f426015cdf0 0x7f42600d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4267ea9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 0x7f42600d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4267ea9640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42600d2590 con 0x7f426015d1d0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4267ea9640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f42600d4310 con 0x7f426015cdf0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4267ea9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f42600d2870 con 0x7f42600d32d0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f426015cdf0 0x7f42600d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f426015cdf0 0x7f42600d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60480/0 (socket says 172.21.15.62:60480) 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 -- 172.21.15.62:0/2480816428 learned_addr learned my addr 172.21.15.62:0/2480816428 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265444640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 0x7f42600d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 -- 172.21.15.62:0/2480816428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 msgr2=0x7f42600d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 0x7f42600d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 -- 172.21.15.62:0/2480816428 >> v1:172.21.15.174:6789/0 conn(0x7f426015d1d0 legacy=0x7f42600b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265c45640 1 -- 172.21.15.62:0/2480816428 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42600b50b0 con 0x7f426015cdf0 2024-04-26T22:28:33.956 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.955+0000 7f4265444640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 0x7f42600d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.956+0000 7f4265c45640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426015cdf0 0x7f42600d2d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f425c009870 tx=0x7f425c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7182f7b1d49d6c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.956+0000 7f4264c43640 1 -- 172.21.15.62:0/2480816428 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c037070 con 0x7f426015cdf0 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.956+0000 7f4264c43640 1 -- 172.21.15.62:0/2480816428 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c032440 con 0x7f426015cdf0 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.956+0000 7f4264c43640 1 -- 172.21.15.62:0/2480816428 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c032810 con 0x7f426015cdf0 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 -- 172.21.15.62:0/2480816428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426015cdf0 msgr2=0x7f42600d2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426015cdf0 0x7f42600d2d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f425c009870 tx=0x7f425c02a210 comp rx=0 tx=0).stop 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 -- 172.21.15.62:0/2480816428 shutdown_connections 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f42600d32d0 0x7f42600d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.958 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 --2- 172.21.15.62:0/2480816428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f426015cdf0 0x7f42600d2d90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 -- 172.21.15.62:0/2480816428 >> 172.21.15.62:0/2480816428 conn(0x7f426008ec30 msgr2=0x7f426009e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 -- 172.21.15.62:0/2480816428 shutdown_connections 2024-04-26T22:28:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.957+0000 7f4267ea9640 1 -- 172.21.15.62:0/2480816428 wait complete. 2024-04-26T22:28:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.958+0000 7f4267ea9640 1 Processor -- start 2024-04-26T22:28:33.959 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.958+0000 7f4267ea9640 1 -- start start 2024-04-26T22:28:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f426015cdf0 0x7f42600b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 0x7f42600b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.961 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f42600b5fc0 con 0x7f42600d32d0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f426015fcc0 con 0x7f426015cdf0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4267ea9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f42600b6140 con 0x7f426015d1d0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60496/0 (socket says 172.21.15.62:60496) 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 -- 172.21.15.62:0/207090866 learned_addr learned my addr 172.21.15.62:0/207090866 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265444640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f426015cdf0 0x7f42600b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4266446640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 0x7f42600b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 -- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 msgr2=0x7f42600b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 0x7f42600b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 -- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f426015cdf0 msgr2=0x7f42600b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f426015cdf0 0x7f42600b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 -- 172.21.15.62:0/207090866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42600bada0 con 0x7f42600d32d0 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4266446640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 0x7f42600b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:33.962 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.959+0000 7f4265c45640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c037040 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c042070 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42600ca020 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c030bb0 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42600ca2c0 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.960+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42600cbb40 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.961+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f425c047460 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.961+0000 7f42567fc640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 0x7f424406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.961+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f425c0ac430 con 0x7f42600d32d0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.962+0000 7f4265444640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 0x7f424406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:33.963 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.962+0000 7f4265444640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 0x7f424406ee70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42600b3fc0 tx=0x7f4250008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:33.966 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:33.966+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42600bada0 con 0x7f42600d32d0 2024-04-26T22:28:34.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.149+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f426006a360 con 0x7f42600d32d0 2024-04-26T22:28:34.150 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.150+0000 7f42567fc640 1 -- 172.21.15.62:0/207090866 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f425c07abc0 con 0x7f42600d32d0 2024-04-26T22:28:34.150 INFO:teuthology.orchestra.run.smithi062.stdout:pg_num: 1 2024-04-26T22:28:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 msgr2=0x7f424406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 0x7f424406ee70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42600b3fc0 tx=0x7f4250008040 comp rx=0 tx=0).stop 2024-04-26T22:28:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 msgr2=0x7f42601536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f425c000c00 tx=0x7f425c0046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 shutdown_connections 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f424406c9b0 0x7f424406ee70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f426015d1d0 0x7f42600b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f426015cdf0 0x7f42600b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 --2- 172.21.15.62:0/207090866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f42600d32d0 0x7f42601536b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 >> 172.21.15.62:0/207090866 conn(0x7f426008ec30 msgr2=0x7f42600d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 shutdown_connections 2024-04-26T22:28:34.153 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.152+0000 7f4267ea9640 1 -- 172.21.15.62:0/207090866 wait complete. 2024-04-26T22:28:34.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T22:28:34.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.263+0000 7fe20b4b0640 1 Processor -- start 2024-04-26T22:28:34.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 -- start start 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2040d3820 0x7fe2040d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe2040b4ba0 0x7fe2040b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2040d0210 con 0x7fe204162b40 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe2040d41c0 con 0x7fe2040b4ba0 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.264+0000 7fe20b4b0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe2040d04f0 con 0x7fe2040d3820 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe20924c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fe204162b40 0x7fe2040d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56574/0 (socket says 172.21.15.62:56574) 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe20924c640 1 -- 172.21.15.62:0/471036274 learned_addr learned my addr 172.21.15.62:0/471036274 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe209a4d640 1 --2- 172.21.15.62:0/471036274 >> v2:172.21.15.62:3300/0 conn(0x7fe2040b4ba0 0x7fe2040b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe208a4b640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2040d3820 0x7fe2040d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe203fff640 1 -- 172.21.15.62:0/471036274 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854080922 0 0) 0x7fe2040d0210 con 0x7fe204162b40 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe203fff640 1 -- 172.21.15.62:0/471036274 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ec003520 con 0x7fe204162b40 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe209a4d640 1 -- 172.21.15.62:0/471036274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2040d3820 msgr2=0x7fe2040d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe209a4d640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2040d3820 0x7fe2040d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe209a4d640 1 -- 172.21.15.62:0/471036274 >> v1:172.21.15.62:6789/0 conn(0x7fe204162b40 legacy=0x7fe2040d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.265+0000 7fe209a4d640 1 -- 172.21.15.62:0/471036274 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2040d07d0 con 0x7fe2040b4ba0 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe209a4d640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040b4ba0 0x7fe2040b5030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009d90 tx=0x7fe1f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a8ec2ebdb2148c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe203fff640 1 -- 172.21.15.62:0/471036274 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4007740 con 0x7fe2040b4ba0 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe203fff640 1 -- 172.21.15.62:0/471036274 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f40078e0 con 0x7fe2040b4ba0 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe203fff640 1 -- 172.21.15.62:0/471036274 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4007c90 con 0x7fe2040b4ba0 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 -- 172.21.15.62:0/471036274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040b4ba0 msgr2=0x7fe2040b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.267 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040b4ba0 0x7fe2040b5030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009d90 tx=0x7fe1f402a660 comp rx=0 tx=0).stop 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 -- 172.21.15.62:0/471036274 shutdown_connections 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040b4ba0 0x7fe2040b5030 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/471036274 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2040d3820 0x7fe2040d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.266+0000 7fe20b4b0640 1 -- 172.21.15.62:0/471036274 >> 172.21.15.62:0/471036274 conn(0x7fe20409e550 msgr2=0x7fe2040d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.267+0000 7fe20b4b0640 1 -- 172.21.15.62:0/471036274 shutdown_connections 2024-04-26T22:28:34.268 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.267+0000 7fe20b4b0640 1 -- 172.21.15.62:0/471036274 wait complete. 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 Processor -- start 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 -- start start 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040b4ba0 0x7fe2041f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 0x7fe2040caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040d12b0 con 0x7fe2040d3820 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040d0d90 con 0x7fe2040b4ba0 2024-04-26T22:28:34.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe20b4b0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe2040cfe30 con 0x7fe204162b40 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.268+0000 7fe208a4b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60518/0 (socket says 172.21.15.62:60518) 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 -- 172.21.15.62:0/760997591 learned_addr learned my addr 172.21.15.62:0/760997591 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe20924c640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040b4ba0 0x7fe2041f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe209a4d640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 0x7fe2040caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 -- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 msgr2=0x7fe2040caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 0x7fe2040caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 -- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040b4ba0 msgr2=0x7fe2041f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040b4ba0 0x7fe2041f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 -- 172.21.15.62:0/760997591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20415ec40 con 0x7fe2040d3820 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe209a4d640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 0x7fe2040caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe208a4b640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d5f0 tx=0x7fe1f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80125f0 con 0x7fe2040d3820 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f8019070 con 0x7fe2040d3820 2024-04-26T22:28:34.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2040cb460 con 0x7fe2040d3820 2024-04-26T22:28:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80129d0 con 0x7fe2040d3820 2024-04-26T22:28:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.269+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2041f51f0 con 0x7fe2040d3820 2024-04-26T22:28:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.270+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20406a360 con 0x7fe2040d3820 2024-04-26T22:28:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.271+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe1f8012b70 con 0x7fe2040d3820 2024-04-26T22:28:34.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.271+0000 7fe201ffb640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 0x7fe1f006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.271+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7fe1f808bbf0 con 0x7fe2040d3820 2024-04-26T22:28:34.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.271+0000 7fe20924c640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 0x7fe1f006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.272+0000 7fe20924c640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 0x7fe1f006ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000f60 tx=0x7fe1fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.275+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe1f805a410 con 0x7fe2040d3820 2024-04-26T22:28:34.460 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.460+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe2040d2960 con 0x7fe2040d3820 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.461+0000 7fe201ffb640 1 -- 172.21.15.62:0/760997591 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7fe1f8059d10 con 0x7fe2040d3820 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: "poolnum": 1, 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: "poolname": ".mgr" 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:34.461 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-26T22:28:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 msgr2=0x7fe1f006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 0x7fe1f006ee40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000f60 tx=0x7fe1fc009040 comp rx=0 tx=0).stop 2024-04-26T22:28:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 msgr2=0x7fe20415e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800d5f0 tx=0x7fe1f800dab0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.463 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 shutdown_connections 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe1f006c980 0x7fe1f006ee40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe204162b40 0x7fe2040caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe2040d3820 0x7fe20415e700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.462+0000 7fe20b4b0640 1 --2- 172.21.15.62:0/760997591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2040b4ba0 0x7fe2041f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.463+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 >> 172.21.15.62:0/760997591 conn(0x7fe20409e550 msgr2=0x7fe2040d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.463+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 shutdown_connections 2024-04-26T22:28:34.465 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.463+0000 7fe20b4b0640 1 -- 172.21.15.62:0/760997591 wait complete. 2024-04-26T22:28:34.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T22:28:34.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T22:28:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 Processor -- start 2024-04-26T22:28:34.579 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 -- start start 2024-04-26T22:28:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 0x7f21680725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21680cfe00 0x7f21680b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2168070c20 con 0x7f216806eed0 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2168070f00 con 0x7f21680cfe00 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.579+0000 7f216df78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f21680d0cd0 con 0x7f216815f200 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2167fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f21680cfe00 0x7f21680b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f21677fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 0x7f21680725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2166ffd640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f216806eed0 0x7f21680cf6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:49428/0 (socket says 172.21.15.62:49428) 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2166ffd640 1 -- 172.21.15.62:0/3074545300 learned_addr learned my addr 172.21.15.62:0/3074545300 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f21667fc640 1 -- 172.21.15.62:0/3074545300 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974707780 0 0) 0x7f2168070c20 con 0x7f216806eed0 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f21667fc640 1 -- 172.21.15.62:0/3074545300 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2154003520 con 0x7f216806eed0 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2167fff640 1 -- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 msgr2=0x7f21680725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2167fff640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 0x7f21680725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2167fff640 1 -- 172.21.15.62:0/3074545300 >> v1:172.21.15.174:6789/0 conn(0x7f216806eed0 legacy=0x7f21680cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.581 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f2167fff640 1 -- 172.21.15.62:0/3074545300 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680d2ac0 con 0x7f21680cfe00 2024-04-26T22:28:34.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.580+0000 7f21677fe640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 0x7f21680725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.581+0000 7f2167fff640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21680cfe00 0x7f21680b1b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2160009d90 tx=0x7f216002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f482fcb5b5344a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.581+0000 7f21667fc640 1 -- 172.21.15.62:0/3074545300 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160007740 con 0x7f21680cfe00 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.581+0000 7f21667fc640 1 -- 172.21.15.62:0/3074545300 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21600078e0 con 0x7f21680cfe00 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.581+0000 7f21667fc640 1 -- 172.21.15.62:0/3074545300 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160007cb0 con 0x7f21680cfe00 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 -- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21680cfe00 msgr2=0x7f21680b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21680cfe00 0x7f21680b1b30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2160009d90 tx=0x7f216002a530 comp rx=0 tx=0).stop 2024-04-26T22:28:34.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 -- 172.21.15.62:0/3074545300 shutdown_connections 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21680cfe00 0x7f21680b1b30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 --2- 172.21.15.62:0/3074545300 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216815f200 0x7f21680725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 -- 172.21.15.62:0/3074545300 >> 172.21.15.62:0/3074545300 conn(0x7f21680d2f60 msgr2=0x7f21680d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 -- 172.21.15.62:0/3074545300 shutdown_connections 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.582+0000 7f216df78640 1 -- 172.21.15.62:0/3074545300 wait complete. 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.583+0000 7f216df78640 1 Processor -- start 2024-04-26T22:28:34.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.583+0000 7f216df78640 1 -- start start 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216806eed0 0x7f216815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 0x7f21680cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21680b21f0 con 0x7f21680cfe00 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21680b2070 con 0x7f216815f200 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f216df78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f21680b2370 con 0x7f216806eed0 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60532/0 (socket says 172.21.15.62:60532) 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 -- 172.21.15.62:0/926103573 learned_addr learned my addr 172.21.15.62:0/926103573 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f21677fe640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216806eed0 0x7f216815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.587 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2167fff640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 0x7f21680cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 -- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216806eed0 msgr2=0x7f216815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216806eed0 0x7f216815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 -- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 msgr2=0x7f21680cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 0x7f21680cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 -- 172.21.15.62:0/926103573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21681604d0 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2167fff640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 0x7f21680cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.584+0000 7f2166ffd640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f215800c4d0 tx=0x7f215800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.585+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158017070 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.585+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2158005ce0 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.585+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21680c7680 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.585+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158007270 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.585+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216815fb20 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.586+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216806a360 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.586+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f215801a070 con 0x7f21680cfe00 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.587+0000 7f215ffff640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 0x7f214806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:34.588 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.587+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f215808abf0 con 0x7f21680cfe00 2024-04-26T22:28:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.587+0000 7f21677fe640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 0x7f214806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:34.589 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.587+0000 7f21677fe640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 0x7f214806ee70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2150009560 tx=0x7f2150009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:34.591 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.591+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2158058c20 con 0x7f21680cfe00 2024-04-26T22:28:34.774 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:34.774+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f21680b8ee0 con 0x7f21680cfe00 2024-04-26T22:28:35.132 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.132+0000 7f215ffff640 1 -- 172.21.15.62:0/926103573 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f2158076e20 con 0x7f21680cfe00 2024-04-26T22:28:35.133 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_metadata' created 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 msgr2=0x7f214806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 0x7f214806ee70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2150009560 tx=0x7f2150009190 comp rx=0 tx=0).stop 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 msgr2=0x7f2168161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f215800c4d0 tx=0x7f215800c990 comp rx=0 tx=0).stop 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 shutdown_connections 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f214806c9b0 0x7f214806ee70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f216815f200 0x7f21680cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21680cfe00 0x7f2168161f60 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 --2- 172.21.15.62:0/926103573 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f216806eed0 0x7f216815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 >> 172.21.15.62:0/926103573 conn(0x7f21680d2f60 msgr2=0x7f21680af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:35.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 shutdown_connections 2024-04-26T22:28:35.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.134+0000 7f216df78640 1 -- 172.21.15.62:0/926103573 wait complete. 2024-04-26T22:28:35.150 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T22:28:35.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.260+0000 7f1d1602a640 1 Processor -- start 2024-04-26T22:28:35.260 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.260+0000 7f1d1602a640 1 -- start start 2024-04-26T22:28:35.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d1602a640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d1015d830 0x7f1d100cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d1602a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1d100cb6f0 0x7f1d100cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d1602a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d1006f750 con 0x7f1d1015ecb0 2024-04-26T22:28:35.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d1602a640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f1d100cbef0 con 0x7f1d1015d830 2024-04-26T22:28:35.263 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d1602a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1d1015e820 con 0x7f1d100cb6f0 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d1015d830 0x7f1d100cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f1d1015d830 0x7f1d100cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60546/0 (socket says 172.21.15.62:60546) 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 -- 172.21.15.62:0/2449412003 learned_addr learned my addr 172.21.15.62:0/2449412003 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0effd640 1 --2- 172.21.15.62:0/2449412003 >> v2:172.21.15.174:3300/0 conn(0x7f1d100cb6f0 0x7f1d100cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 -- 172.21.15.62:0/2449412003 >> v2:172.21.15.174:3300/0 conn(0x7f1d100cb6f0 msgr2=0x7f1d100cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 --2- 172.21.15.62:0/2449412003 >> v2:172.21.15.174:3300/0 conn(0x7f1d100cb6f0 0x7f1d100cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 -- 172.21.15.62:0/2449412003 >> v1:172.21.15.174:6789/0 conn(0x7f1d1015ecb0 legacy=0x7f1d1015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.261+0000 7f1d0f7fe640 1 -- 172.21.15.62:0/2449412003 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1006fa30 con 0x7f1d1015d830 2024-04-26T22:28:35.264 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d0f7fe640 1 --2- 172.21.15.62:0/2449412003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015d830 0x7f1d100cc0f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1d10068d30 tx=0x7f1d0002c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0f120f269cfdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d0e7fc640 1 -- 172.21.15.62:0/2449412003 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00038070 con 0x7f1d1015d830 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d0e7fc640 1 -- 172.21.15.62:0/2449412003 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d00031710 con 0x7f1d1015d830 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d0e7fc640 1 -- 172.21.15.62:0/2449412003 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00031ae0 con 0x7f1d1015d830 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d1602a640 1 -- 172.21.15.62:0/2449412003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015d830 msgr2=0x7f1d100cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.262+0000 7f1d1602a640 1 --2- 172.21.15.62:0/2449412003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015d830 0x7f1d100cc0f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1d10068d30 tx=0x7f1d0002c480 comp rx=0 tx=0).stop 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 -- 172.21.15.62:0/2449412003 shutdown_connections 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 --2- 172.21.15.62:0/2449412003 >> v2:172.21.15.174:3300/0 conn(0x7f1d100cb6f0 0x7f1d100cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 --2- 172.21.15.62:0/2449412003 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015d830 0x7f1d100cc0f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 -- 172.21.15.62:0/2449412003 >> 172.21.15.62:0/2449412003 conn(0x7f1d100d0d80 msgr2=0x7f1d100d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:35.265 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 -- 172.21.15.62:0/2449412003 shutdown_connections 2024-04-26T22:28:35.266 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.263+0000 7f1d1602a640 1 -- 172.21.15.62:0/2449412003 wait complete. 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.264+0000 7f1d1602a640 1 Processor -- start 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.264+0000 7f1d1602a640 1 -- start start 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d100cb6f0 0x7f1d100c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1015d830 0x7f1d10152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1d100ce950 con 0x7f1d1015ecb0 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1006dfb0 con 0x7f1d1015d830 2024-04-26T22:28:35.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d1602a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1d100cead0 con 0x7f1d100cb6f0 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60556/0 (socket says 172.21.15.62:60556) 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 -- 172.21.15.62:0/1343275802 learned_addr learned my addr 172.21.15.62:0/1343275802 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0effd640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1015d830 0x7f1d10152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0f7fe640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d100cb6f0 0x7f1d100c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 -- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d100cb6f0 msgr2=0x7f1d100c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d100cb6f0 0x7f1d100c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 -- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1015d830 msgr2=0x7f1d10152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1015d830 0x7f1d10152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 -- 172.21.15.62:0/1343275802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d100b6b90 con 0x7f1d1015ecb0 2024-04-26T22:28:35.270 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1d0ffff640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c6b0 tx=0x7f1d0800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.265+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08003b70 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.266+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d08003d30 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.266+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d100b3a00 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.266+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08005310 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.266+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d100b3fc0 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.267+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d1006a360 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.267+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f1d080054b0 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.268+0000 7f1cf7fff640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 0x7f1cfc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.268+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7f1d0808b650 con 0x7f1d1015ecb0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.268+0000 7f1d0f7fe640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 0x7f1cfc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:35.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.269+0000 7f1d0f7fe640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 0x7f1cfc06ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d00035000 tx=0x7f1d00008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:35.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.273+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d080596b0 con 0x7f1d1015ecb0 2024-04-26T22:28:35.457 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:35.457+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1d1015e100 con 0x7f1d1015ecb0 2024-04-26T22:28:36.140 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.139+0000 7f1cf7fff640 1 -- 172.21.15.62:0/1343275802 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7f1d0805d9b0 con 0x7f1d1015ecb0 2024-04-26T22:28:36.140 INFO:teuthology.orchestra.run.smithi062.stderr:pool 'cephfs_data' created 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 msgr2=0x7f1cfc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 0x7f1cfc06ee40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d00035000 tx=0x7f1d00008030 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 msgr2=0x7f1d101527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800c6b0 tx=0x7f1d0800cb70 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 shutdown_connections 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f1cfc06c980 0x7f1cfc06ee40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1d1015ecb0 0x7f1d101527b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1015d830 0x7f1d10152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 --2- 172.21.15.62:0/1343275802 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d100cb6f0 0x7f1d100c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.142+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 >> 172.21.15.62:0/1343275802 conn(0x7f1d100d0d80 msgr2=0x7f1d100afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.143+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 shutdown_connections 2024-04-26T22:28:36.143 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.143+0000 7f1d1602a640 1 -- 172.21.15.62:0/1343275802 wait complete. 2024-04-26T22:28:36.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T22:28:36.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 Processor -- start 2024-04-26T22:28:36.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 -- start start 2024-04-26T22:28:36.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91fc160800 0x7f91fc1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 0x7f91fc0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91fc0d0930 con 0x7f91fc0d5b60 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f91fc0d0c00 con 0x7f91fc160800 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.274+0000 7f9201cbb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc0d0ee0 con 0x7f91fc0d2320 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91fc160800 0x7f91fc1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f91fc160800 0x7f91fc1610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60572/0 (socket says 172.21.15.62:60572) 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 -- 172.21.15.62:0/3340577012 learned_addr learned my addr 172.21.15.62:0/3340577012 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91faffd640 1 --1- 172.21.15.62:0/3340577012 >> v1:172.21.15.62:6789/0 conn(0x7f91fc0d5b60 0x7f91fc0d1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56586/0 (socket says 172.21.15.62:56586) 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 -- 172.21.15.62:0/3340577012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 msgr2=0x7f91fc0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fbfff640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 0x7f91fc0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 0x7f91fc0d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fa7fc640 1 -- 172.21.15.62:0/3340577012 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455083341 0 0) 0x7f91fc0d0930 con 0x7f91fc0d5b60 2024-04-26T22:28:36.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 -- 172.21.15.62:0/3340577012 >> v1:172.21.15.62:6789/0 conn(0x7f91fc0d5b60 legacy=0x7f91fc0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fb7fe640 1 -- 172.21.15.62:0/3340577012 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc0d11c0 con 0x7f91fc160800 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.275+0000 7f91fbfff640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 0x7f91fc0d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc1610c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009870 tx=0x7f91ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c68588bfb7a9171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f91fa7fc640 1 -- 172.21.15.62:0/3340577012 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec037070 con 0x7f91fc160800 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f91fa7fc640 1 -- 172.21.15.62:0/3340577012 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ec032440 con 0x7f91fc160800 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f91fa7fc640 1 -- 172.21.15.62:0/3340577012 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec032810 con 0x7f91fc160800 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 -- 172.21.15.62:0/3340577012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 msgr2=0x7f91fc1610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc1610c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91ec009870 tx=0x7f91ec02a270 comp rx=0 tx=0).stop 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 -- 172.21.15.62:0/3340577012 shutdown_connections 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d2320 0x7f91fc0d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 --2- 172.21.15.62:0/3340577012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc1610c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 -- 172.21.15.62:0/3340577012 >> 172.21.15.62:0/3340577012 conn(0x7f91fc09e550 msgr2=0x7f91fc0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:36.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 -- 172.21.15.62:0/3340577012 shutdown_connections 2024-04-26T22:28:36.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.276+0000 7f9201cbb640 1 -- 172.21.15.62:0/3340577012 wait complete. 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 Processor -- start 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 -- start start 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 0x7f91fc15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d5b60 0x7f91fc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc0b6300 con 0x7f91fc160800 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.278+0000 7f9201cbb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc0d30f0 con 0x7f91fc0d2320 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f9201cbb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f91fc160be0 con 0x7f91fc0d5b60 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60588/0 (socket says 172.21.15.62:60588) 2024-04-26T22:28:36.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 -- 172.21.15.62:0/961078731 learned_addr learned my addr 172.21.15.62:0/961078731 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91faffd640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d5b60 0x7f91fc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 0x7f91fc15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 -- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d5b60 msgr2=0x7f91fc0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d5b60 0x7f91fc0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 -- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 msgr2=0x7f91fc15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 0x7f91fc15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 -- 172.21.15.62:0/961078731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc159c20 con 0x7f91fc160800 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 0x7f91fc15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91fbfff640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91f400cdd0 tx=0x7f91f400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f401a070 con 0x7f91fc160800 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f400d840 con 0x7f91fc160800 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f400dc70 con 0x7f91fc160800 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.279+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc159f00 con 0x7f91fc160800 2024-04-26T22:28:36.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.280+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc0b5630 con 0x7f91fc160800 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.280+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc06a360 con 0x7f91fc160800 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.281+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f91f401d070 con 0x7f91fc160800 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.282+0000 7f91e3fff640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 0x7f91e806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.282+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7f91f408c2e0 con 0x7f91fc160800 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.282+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 0x7f91e806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:36.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.282+0000 7f91fb7fe640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 0x7f91e806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034000 tx=0x7f91ec02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:36.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.286+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91f405a780 con 0x7f91fc160800 2024-04-26T22:28:36.513 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:36.513+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f91fc15fa90 con 0x7f91fc160800 2024-04-26T22:28:37.141 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.141+0000 7f91e3fff640 1 -- 172.21.15.62:0/961078731 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) ==== 172+0+0 (secure 0 0 0) 0x7f91f405a080 con 0x7f91fc160800 2024-04-26T22:28:37.142 INFO:teuthology.orchestra.run.smithi062.stderr:new fs with metadata pool 6 and data pool 7 2024-04-26T22:28:37.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 msgr2=0x7f91e806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 0x7f91e806ee40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f91ec034000 tx=0x7f91ec02e000 comp rx=0 tx=0).stop 2024-04-26T22:28:37.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 msgr2=0x7f91fc0b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f91f400cdd0 tx=0x7f91f400a710 comp rx=0 tx=0).stop 2024-04-26T22:28:37.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 shutdown_connections 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f91e806c980 0x7f91e806ee40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f91fc160800 0x7f91fc0b50f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f91fc0d5b60 0x7f91fc0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 --2- 172.21.15.62:0/961078731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91fc0d2320 0x7f91fc15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 >> 172.21.15.62:0/961078731 conn(0x7f91fc09e550 msgr2=0x7f91fc160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 shutdown_connections 2024-04-26T22:28:37.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.144+0000 7f9201cbb640 1 -- 172.21.15.62:0/961078731 wait complete. 2024-04-26T22:28:37.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.276+0000 7fe6f93bf640 1 Processor -- start 2024-04-26T22:28:37.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.276+0000 7fe6f93bf640 1 -- start start 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6f415f380 0x7fe6f40d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe6f415f760 0x7fe6f40d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f40cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40d43a0 con 0x7fe6f415f380 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40d3de0 con 0x7fe6f415f760 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f93bf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40d40c0 con 0x7fe6f40d5b50 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6f415f380 0x7fe6f40d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe6f415f380 0x7fe6f40d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60592/0 (socket says 172.21.15.62:60592) 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 -- 172.21.15.62:0/728600539 learned_addr learned my addr 172.21.15.62:0/728600539 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f37fe640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f40cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f27fc640 1 --2- 172.21.15.62:0/728600539 >> v2:172.21.15.174:3300/0 conn(0x7fe6f415f760 0x7fe6f40d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 -- 172.21.15.62:0/728600539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 msgr2=0x7fe6f40cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f40cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 -- 172.21.15.62:0/728600539 >> v2:172.21.15.174:3300/0 conn(0x7fe6f415f760 msgr2=0x7fe6f40d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/728600539 >> v2:172.21.15.174:3300/0 conn(0x7fe6f415f760 0x7fe6f40d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.277+0000 7fe6f2ffd640 1 -- 172.21.15.62:0/728600539 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40b8280 con 0x7fe6f415f380 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f380 0x7fe6f40d1bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe6f408d560 tx=0x7fe6e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c49ad4ae123a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f1ffb640 1 -- 172.21.15.62:0/728600539 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4037070 con 0x7fe6f415f380 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f1ffb640 1 -- 172.21.15.62:0/728600539 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e4032440 con 0x7fe6f415f380 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f1ffb640 1 -- 172.21.15.62:0/728600539 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4032810 con 0x7fe6f415f380 2024-04-26T22:28:37.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 -- 172.21.15.62:0/728600539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f380 msgr2=0x7fe6f40d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f380 0x7fe6f40d1bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe6f408d560 tx=0x7fe6e402a270 comp rx=0 tx=0).stop 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 -- 172.21.15.62:0/728600539 shutdown_connections 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f40cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/728600539 >> v2:172.21.15.174:3300/0 conn(0x7fe6f415f760 0x7fe6f40d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.278+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/728600539 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f380 0x7fe6f40d1bd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.279+0000 7fe6f93bf640 1 -- 172.21.15.62:0/728600539 >> 172.21.15.62:0/728600539 conn(0x7fe6f409e560 msgr2=0x7fe6f40d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.279+0000 7fe6f93bf640 1 -- 172.21.15.62:0/728600539 shutdown_connections 2024-04-26T22:28:37.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.279+0000 7fe6f93bf640 1 -- 172.21.15.62:0/728600539 wait complete. 2024-04-26T22:28:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.280+0000 7fe6f93bf640 1 Processor -- start 2024-04-26T22:28:37.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.280+0000 7fe6f93bf640 1 -- start start 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f41f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6f415f380 0x7fe6f409efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40d1280 con 0x7fe6f415f760 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40d1100 con 0x7fe6f415f380 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40d3840 con 0x7fe6f40d5b50 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60600/0 (socket says 172.21.15.62:60600) 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 -- 172.21.15.62:0/3197469879 learned_addr learned my addr 172.21.15.62:0/3197469879 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f41f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f27fc640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6f415f380 0x7fe6f409efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 -- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 msgr2=0x7fe6f41f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f41f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 -- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6f415f380 msgr2=0x7fe6f409efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6f415f380 0x7fe6f409efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 -- 172.21.15.62:0/3197469879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f41f9c00 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f37fe640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec007e60 tx=0x7fe6ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec014620 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6ec019070 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f40ca730 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec014a20 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.281+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f40cacf0 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.282+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f40d1bd0 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.283+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe6ec014bc0 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.283+0000 7fe6db7fe640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 0x7fe6e006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.284+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fe6ec08c420 con 0x7fe6f415f760 2024-04-26T22:28:37.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.284+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 0x7fe6e006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.286 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.284+0000 7fe6f2ffd640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 0x7fe6e006ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4034000 tx=0x7fe6e402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.288+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6f40d1bd0 con 0x7fe6f415f760 2024-04-26T22:28:37.473 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.473+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6f415ef30 con 0x7fe6f415f760 2024-04-26T22:28:37.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.475+0000 7fe6db7fe640 1 -- 172.21.15.62:0/3197469879 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16626 (secure 0 0 0) 0x7fe6f415ef30 con 0x7fe6f415f760 2024-04-26T22:28:37.475 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:37.475 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":42,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:36.514591+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:34.778483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:35.461326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.476+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 msgr2=0x7fe6e006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.476+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 0x7fe6e006ee40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4034000 tx=0x7fe6e402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.476+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 msgr2=0x7fe6f409f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.476+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec007e60 tx=0x7fe6ec00e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 shutdown_connections 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe6e006c980 0x7fe6e006ee40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6f415f760 0x7fe6f409f510 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6f415f380 0x7fe6f409efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 --2- 172.21.15.62:0/3197469879 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe6f40d5b50 0x7fe6f41f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 >> 172.21.15.62:0/3197469879 conn(0x7fe6f409e560 msgr2=0x7fe6f40b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 shutdown_connections 2024-04-26T22:28:37.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.477+0000 7fe6f93bf640 1 -- 172.21.15.62:0/3197469879 wait complete. 2024-04-26T22:28:37.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.589+0000 7f6f97d91640 1 Processor -- start 2024-04-26T22:28:37.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 -- start start 2024-04-26T22:28:37.600 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f900d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6f900b4ba0 0x7f6f900b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f900d0210 con 0x7f6f90162b40 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f6f900d41c0 con 0x7f6f900b4ba0 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.590+0000 7f6f97d91640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6f900d04f0 con 0x7f6f900d3820 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f95b2d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f6f90162b40 0x7f6f900d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56590/0 (socket says 172.21.15.62:56590) 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f95b2d640 1 -- 172.21.15.62:0/1923551554 learned_addr learned my addr 172.21.15.62:0/1923551554 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085933116 0 0) 0x7f6f900d0210 con 0x7f6f90162b40 2024-04-26T22:28:37.601 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f9532c640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f900d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f7c003520 con 0x7f6f90162b40 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f9632e640 1 --2- 172.21.15.62:0/1923551554 >> v2:172.21.15.174:3300/0 conn(0x7f6f900b4ba0 0x7f6f900b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3445133190 0 0) 0x7f6f7c003520 con 0x7f6f90162b40 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 msgr2=0x7f6f900d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f900d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 >> v2:172.21.15.174:3300/0 conn(0x7f6f900b4ba0 msgr2=0x7f6f900b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.602 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 --2- 172.21.15.62:0/1923551554 >> v2:172.21.15.174:3300/0 conn(0x7f6f900b4ba0 0x7f6f900b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f900d07d0 con 0x7f6f90162b40 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f6f8c001670 con 0x7f6f90162b40 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f9532c640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f900d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 >> v1:172.21.15.62:6789/0 conn(0x7f6f90162b40 legacy=0x7f6f900d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7c0039b0 0x7f6f7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f7c004d40 0x7f6f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.603 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900d04f0 con 0x7f6f7c0039b0 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c00b9c0 con 0x7f6f7c004d40 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c00bb40 con 0x7f6f900d3820 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7c0039b0 0x7f6f7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f9532c640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.591+0000 7f6f9632e640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f7c004d40 0x7f6f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 -- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 msgr2=0x7f6f7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.604 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f900d3820 0x7f6f7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 -- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f7c004d40 msgr2=0x7f6f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f7c004d40 0x7f6f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 -- 172.21.15.62:0/1923551554 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c003520 con 0x7f6f7c0039b0 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7c0039b0 0x7f6f7c003dd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00c210 tx=0x7f6f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3214c06a1876934f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c007d50 con 0x7f6f7c0039b0 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8c005af0 con 0x7f6f7c0039b0 2024-04-26T22:28:37.605 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.592+0000 7f6f94b2b640 1 -- 172.21.15.62:0/1923551554 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c007070 con 0x7f6f7c0039b0 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 -- 172.21.15.62:0/1923551554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7c0039b0 msgr2=0x7f6f7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 --2- 172.21.15.62:0/1923551554 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f7c0039b0 0x7f6f7c003dd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00c210 tx=0x7f6f7c005830 comp rx=0 tx=0).stop 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f9632e640 1 -- 172.21.15.62:0/1923551554 reap_dead start 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 -- 172.21.15.62:0/1923551554 shutdown_connections 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 -- 172.21.15.62:0/1923551554 >> 172.21.15.62:0/1923551554 conn(0x7f6f9009e550 msgr2=0x7f6f900d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 -- 172.21.15.62:0/1923551554 shutdown_connections 2024-04-26T22:28:37.606 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.593+0000 7f6f97d91640 1 -- 172.21.15.62:0/1923551554 wait complete. 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.594+0000 7f6f97d91640 1 Processor -- start 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.594+0000 7f6f97d91640 1 -- start start 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f7c0039b0 0x7f6f901f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 0x7f6f900d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900d12b0 con 0x7f6f90154bc0 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900d0d90 con 0x7f6f900d1430 2024-04-26T22:28:37.607 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f97d91640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6f900cfe30 con 0x7f6f7c0039b0 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60612/0 (socket says 172.21.15.62:60612) 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 -- 172.21.15.62:0/922751390 learned_addr learned my addr 172.21.15.62:0/922751390 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f7c0039b0 0x7f6f901f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9632e640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 0x7f6f900d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 -- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f7c0039b0 msgr2=0x7f6f901f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.608 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f7c0039b0 0x7f6f901f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 -- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 msgr2=0x7f6f900d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 0x7f6f900d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9532c640 1 -- 172.21.15.62:0/922751390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f90160ab0 con 0x7f6f90154bc0 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.595+0000 7f6f9632e640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 0x7f6f900d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f9532c640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6f800115f0 tx=0x7f6f80011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80019070 con 0x7f6f90154bc0 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f80004220 con 0x7f6f90154bc0 2024-04-26T22:28:37.609 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f90160d90 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000f040 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.596+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f90161350 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.597+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9006a360 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.597+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6f8000f6f0 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.598+0000 7f6f867fc640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 0x7f6f7406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.598+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8008bf20 con 0x7f6f90154bc0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.598+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 0x7f6f7406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.610 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.599+0000 7f6f95b2d640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 0x7f6f7406ee70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c033000 tx=0x7f6f8c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.611 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.602+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f80059b70 con 0x7f6f90154bc0 2024-04-26T22:28:37.788 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.788+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f900d2960 con 0x7f6f90154bc0 2024-04-26T22:28:37.789 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.789+0000 7f6f867fc640 1 -- 172.21.15.62:0/922751390 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16626 (secure 0 0 0) 0x7f6f8005de70 con 0x7f6f90154bc0 2024-04-26T22:28:37.790 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:37.790 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":42,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:36.514591+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:34.778483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:35.461326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 msgr2=0x7f6f7406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 0x7f6f7406ee70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c033000 tx=0x7f6f8c009030 comp rx=0 tx=0).stop 2024-04-26T22:28:37.792 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 msgr2=0x7f6f901578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6f800115f0 tx=0x7f6f80011ab0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 shutdown_connections 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f6f7406c9b0 0x7f6f7406ee70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f900d1430 0x7f6f900d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6f90154bc0 0x7f6f901578f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 --2- 172.21.15.62:0/922751390 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6f7c0039b0 0x7f6f901f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 >> 172.21.15.62:0/922751390 conn(0x7f6f9009e550 msgr2=0x7f6f900d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.793 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 shutdown_connections 2024-04-26T22:28:37.794 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.791+0000 7f6f97d91640 1 -- 172.21.15.62:0/922751390 wait complete. 2024-04-26T22:28:37.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T22:28:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.907+0000 7fb66dbf0640 1 Processor -- start 2024-04-26T22:28:37.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 -- start start 2024-04-26T22:28:37.909 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb6680d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6680d2150 0x7fb6680d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb668161510 con 0x7fb668160800 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fb6680d2590 con 0x7fb6680d2150 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb66dbf0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680d0930 con 0x7fb6681610c0 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb667fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6680d2150 0x7fb6680d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.908+0000 7fb667fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fb6680d2150 0x7fb6680d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60626/0 (socket says 172.21.15.62:60626) 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 -- 172.21.15.62:0/41685995 learned_addr learned my addr 172.21.15.62:0/41685995 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb666ffd640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb6680d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 -- 172.21.15.62:0/41685995 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 msgr2=0x7fb6680d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb6680d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.910 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 -- 172.21.15.62:0/41685995 >> v1:172.21.15.174:6789/0 conn(0x7fb668160800 legacy=0x7fb6680d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 -- 172.21.15.62:0/41685995 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680d0c00 con 0x7fb6680d2150 2024-04-26T22:28:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.909+0000 7fb667fff640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6680d2150 0x7fb6680d3610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb660009e50 tx=0x7fb66002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8547066f5ed4fd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb6667fc640 1 -- 172.21.15.62:0/41685995 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660036070 con 0x7fb6680d2150 2024-04-26T22:28:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb6667fc640 1 -- 172.21.15.62:0/41685995 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb660031600 con 0x7fb6680d2150 2024-04-26T22:28:37.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb6667fc640 1 -- 172.21.15.62:0/41685995 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600319d0 con 0x7fb6680d2150 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 -- 172.21.15.62:0/41685995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6680d2150 msgr2=0x7fb6680d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6680d2150 0x7fb6680d3610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb660009e50 tx=0x7fb66002a710 comp rx=0 tx=0).stop 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 -- 172.21.15.62:0/41685995 shutdown_connections 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6680d2150 0x7fb6680d3610 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/41685995 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb6680d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 -- 172.21.15.62:0/41685995 >> 172.21.15.62:0/41685995 conn(0x7fb66809e550 msgr2=0x7fb6680d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 -- 172.21.15.62:0/41685995 shutdown_connections 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.910+0000 7fb66dbf0640 1 -- 172.21.15.62:0/41685995 wait complete. 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.911+0000 7fb66dbf0640 1 Processor -- start 2024-04-26T22:28:37.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 -- start start 2024-04-26T22:28:37.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 0x7fb668156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb66815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680d2e10 con 0x7fb668160800 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680d11c0 con 0x7fb6680d2150 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb66dbf0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680d30f0 con 0x7fb6681610c0 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb666ffd640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60630/0 (socket says 172.21.15.62:60630) 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.912+0000 7fb666ffd640 1 -- 172.21.15.62:0/272606575 learned_addr learned my addr 172.21.15.62:0/272606575 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb667fff640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb66815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb6677fe640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 0x7fb668156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 -- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 msgr2=0x7fb66815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb66815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 -- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 msgr2=0x7fb668156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 0x7fb668156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 -- 172.21.15.62:0/272606575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6680b49a0 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb6677fe640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 0x7fb668156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb666ffd640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00e870 tx=0x7fb65c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c017070 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c00c940 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00cd30 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6680b4c80 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.913+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6680b5240 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.914+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6680b5680 con 0x7fb668160800 2024-04-26T22:28:37.916 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.915+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb65c01a070 con 0x7fb668160800 2024-04-26T22:28:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.915+0000 7fb64ffff640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.915+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fb65c08bc70 con 0x7fb668160800 2024-04-26T22:28:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.915+0000 7fb6677fe640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:37.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.916+0000 7fb6677fe640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 0x7fb65406ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb66815af40 tx=0x7fb658009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:37.919 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:37.919+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb65c059af0 con 0x7fb668160800 2024-04-26T22:28:38.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.142+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb6680d4530 con 0x7fb668160800 2024-04-26T22:28:38.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.157+0000 7fb64ffff640 1 -- 172.21.15.62:0/272606575 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7fb65c05ddf0 con 0x7fb668160800 2024-04-26T22:28:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 msgr2=0x7fb65406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 0x7fb65406ee40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb66815af40 tx=0x7fb658009040 comp rx=0 tx=0).stop 2024-04-26T22:28:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 msgr2=0x7fb6680ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:38.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00e870 tx=0x7fb65c00ed30 comp rx=0 tx=0).stop 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 shutdown_connections 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb65406c980 0x7fb65406ee40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6681610c0 0x7fb66815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb668160800 0x7fb6680ccd20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 --2- 172.21.15.62:0/272606575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6680d2150 0x7fb668156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 >> 172.21.15.62:0/272606575 conn(0x7fb66809e550 msgr2=0x7fb66815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 shutdown_connections 2024-04-26T22:28:38.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.159+0000 7fb66dbf0640 1 -- 172.21.15.62:0/272606575 wait complete. 2024-04-26T22:28:38.170 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T22:28:38.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T22:28:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.274+0000 7fd0f9209640 1 Processor -- start 2024-04-26T22:28:38.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.274+0000 7fd0f9209640 1 -- start start 2024-04-26T22:28:38.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f9209640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f415f790 0x7fd0f40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f9209640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0f40cb890 0x7fd0f40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f9209640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f40cd430 con 0x7fd0f40b2250 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f9209640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f40cbe50 con 0x7fd0f40cb890 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f9209640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f40cc130 con 0x7fd0f415f790 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0f40cb890 0x7fd0f40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fd0f40cb890 0x7fd0f40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60632/0 (socket says 172.21.15.62:60632) 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 -- 172.21.15.62:0/3584268630 learned_addr learned my addr 172.21.15.62:0/3584268630 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0f2d9d640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f415f790 0x7fd0f40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 -- 172.21.15.62:0/3584268630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f415f790 msgr2=0x7fd0f40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f415f790 0x7fd0f40ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 -- 172.21.15.62:0/3584268630 >> v1:172.21.15.174:6789/0 conn(0x7fd0f40b2250 legacy=0x7fd0f40b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.275+0000 7fd0ebfff640 1 -- 172.21.15.62:0/3584268630 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f40cc410 con 0x7fd0f40cb890 2024-04-26T22:28:38.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0ebfff640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ccef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009870 tx=0x7fd0dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e5d70ab97b36cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f259c640 1 -- 172.21.15.62:0/3584268630 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc037070 con 0x7fd0f40cb890 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f259c640 1 -- 172.21.15.62:0/3584268630 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0dc032440 con 0x7fd0f40cb890 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f259c640 1 -- 172.21.15.62:0/3584268630 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc032810 con 0x7fd0f40cb890 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 -- 172.21.15.62:0/3584268630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f40cb890 msgr2=0x7fd0f40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ccef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009870 tx=0x7fd0dc02a210 comp rx=0 tx=0).stop 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 -- 172.21.15.62:0/3584268630 shutdown_connections 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ccef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 --2- 172.21.15.62:0/3584268630 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f415f790 0x7fd0f40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.276+0000 7fd0f9209640 1 -- 172.21.15.62:0/3584268630 >> 172.21.15.62:0/3584268630 conn(0x7fd0f40737f0 msgr2=0x7fd0f4073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.277+0000 7fd0f9209640 1 -- 172.21.15.62:0/3584268630 shutdown_connections 2024-04-26T22:28:38.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.277+0000 7fd0f9209640 1 -- 172.21.15.62:0/3584268630 wait complete. 2024-04-26T22:28:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.278+0000 7fd0f9209640 1 Processor -- start 2024-04-26T22:28:38.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.278+0000 7fd0f9209640 1 -- start start 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f40b2250 0x7fd0f415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40d0ba0 con 0x7fd0f415f790 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40cc9d0 con 0x7fd0f40cb890 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f40b47e0 con 0x7fd0f40b2250 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60636/0 (socket says 172.21.15.62:60636) 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 -- 172.21.15.62:0/1536445280 learned_addr learned my addr 172.21.15.62:0/1536445280 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f2d9d640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f40b2250 0x7fd0f415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0ebfff640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 -- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f40b2250 msgr2=0x7fd0f415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:38.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f40b2250 0x7fd0f415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 -- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f40cb890 msgr2=0x7fd0f40ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 -- 172.21.15.62:0/1536445280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f40c92d0 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f359e640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c7b0 tx=0x7fd0ec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec004b60 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec004d00 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f40c9570 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec00aa90 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.279+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f40c9e10 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.280+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f406a360 con 0x7fd0f415f790 2024-04-26T22:28:38.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.281+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fd0ec00ac30 con 0x7fd0f415f790 2024-04-26T22:28:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.281+0000 7fd0eb7fe640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 0x7fd0d406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.281+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec08bec0 con 0x7fd0f415f790 2024-04-26T22:28:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.282+0000 7fd0f2d9d640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 0x7fd0d406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:38.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.282+0000 7fd0f2d9d640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 0x7fd0d406ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40079a0 tx=0x7fd0e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:38.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.286+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd0ec059d40 con 0x7fd0f415f790 2024-04-26T22:28:38.512 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:38.512+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd0f40c9740 con 0x7fd0f415f790 2024-04-26T22:28:39.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.164+0000 7fd0eb7fe640 1 -- 172.21.15.62:0/1536445280 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7fd0ec05e040 con 0x7fd0f415f790 2024-04-26T22:28:39.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 msgr2=0x7fd0d406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 0x7fd0d406ee40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40079a0 tx=0x7fd0e4008040 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 msgr2=0x7fd0f40ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c7b0 tx=0x7fd0ec00cc70 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 shutdown_connections 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fd0d406c980 0x7fd0d406ee40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f415f790 0x7fd0f40ca870 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0f40cb890 0x7fd0f40ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 --2- 172.21.15.62:0/1536445280 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0f40b2250 0x7fd0f415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 >> 172.21.15.62:0/1536445280 conn(0x7fd0f40737f0 msgr2=0x7fd0f415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 shutdown_connections 2024-04-26T22:28:39.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.166+0000 7fd0f9209640 1 -- 172.21.15.62:0/1536445280 wait complete. 2024-04-26T22:28:39.179 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T22:28:39.179 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:39.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.283+0000 7f4c6b847640 1 Processor -- start 2024-04-26T22:28:39.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 -- start start 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4c64162b40 0x7f4c640d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c640d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c640d0210 con 0x7f4c640d1850 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4c640d41c0 con 0x7f4c64162b40 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.284+0000 7f4c6b847640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4c640d04f0 con 0x7f4c640d3820 2024-04-26T22:28:39.287 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c69de4640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f4c640d1850 0x7f4c640d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56598/0 (socket says 172.21.15.62:56598) 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c69de4640 1 -- 172.21.15.62:0/2765621461 learned_addr learned my addr 172.21.15.62:0/2765621461 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c640d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c695e3640 1 --2- 172.21.15.62:0/2765621461 >> v2:172.21.15.174:3300/0 conn(0x7f4c64162b40 0x7f4c640d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253552392 0 0) 0x7f4c640d0210 con 0x7f4c640d1850 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c4c003520 con 0x7f4c640d1850 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2475030209 0 0) 0x7f4c4c003520 con 0x7f4c640d1850 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 msgr2=0x7f4c640d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c640d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 >> v2:172.21.15.174:3300/0 conn(0x7f4c64162b40 msgr2=0x7f4c640d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 --2- 172.21.15.62:0/2765621461 >> v2:172.21.15.174:3300/0 conn(0x7f4c64162b40 0x7f4c640d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c640d07d0 con 0x7f4c640d1850 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f4c54001670 con 0x7f4c640d1850 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c640d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T22:28:39.288 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 >> v1:172.21.15.62:6789/0 conn(0x7f4c640d1850 legacy=0x7f4c640d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640d04f0 con 0x7f4c4c004800 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c00b9c0 con 0x7f4c4c0039b0 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c00bb40 con 0x7f4c640d3820 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.285+0000 7f4c695e3640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c69de4640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 -- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 msgr2=0x7f4c4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640d3820 0x7f4c4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 -- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c0039b0 msgr2=0x7f4c4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c0039b0 0x7f4c4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.289 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 -- 172.21.15.62:0/2765621461 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c003520 con 0x7f4c4c004800 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.286+0000 7f4c68de2640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009c90 tx=0x7f4c4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fdb00c006ccacb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58036070 con 0x7f4c4c004800 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c58031400 con 0x7f4c4c004800 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c63fff640 1 -- 172.21.15.62:0/2765621461 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58009860 con 0x7f4c4c004800 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 -- 172.21.15.62:0/2765621461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c004800 msgr2=0x7f4c4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 --2- 172.21.15.62:0/2765621461 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c4c004800 0x7f4c4c004c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009c90 tx=0x7f4c4c005830 comp rx=0 tx=0).stop 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c69de4640 1 -- 172.21.15.62:0/2765621461 reap_dead start 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 -- 172.21.15.62:0/2765621461 shutdown_connections 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 -- 172.21.15.62:0/2765621461 >> 172.21.15.62:0/2765621461 conn(0x7f4c6409e550 msgr2=0x7f4c640d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 -- 172.21.15.62:0/2765621461 shutdown_connections 2024-04-26T22:28:39.290 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.287+0000 7f4c6b847640 1 -- 172.21.15.62:0/2765621461 wait complete. 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.288+0000 7f4c6b847640 1 Processor -- start 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.288+0000 7f4c6b847640 1 -- start start 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c004800 0x7f4c640c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640cdd80 0x7f4c640ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640b5bb0 con 0x7f4c641582c0 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c640d0d90 con 0x7f4c4c004800 2024-04-26T22:28:39.293 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c6b847640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4c640d6110 con 0x7f4c640cdd80 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c68de2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c68de2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60668/0 (socket says 172.21.15.62:60668) 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c68de2640 1 -- 172.21.15.62:0/1138013897 learned_addr learned my addr 172.21.15.62:0/1138013897 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c69de4640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640cdd80 0x7f4c640ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.289+0000 7f4c695e3640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c004800 0x7f4c640c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 -- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640cdd80 msgr2=0x7f4c640ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640cdd80 0x7f4c640ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 -- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c004800 msgr2=0x7f4c640c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c004800 0x7f4c640c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 -- 172.21.15.62:0/1138013897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c641f6590 con 0x7f4c641582c0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c68de2640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c58000c00 tx=0x7f4c58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58040070 con 0x7f4c641582c0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c580093e0 con 0x7f4c641582c0 2024-04-26T22:28:39.294 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c640c8fd0 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c58036040 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.290+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c640c9590 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.291+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c640d0210 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.292+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f4c58043070 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.292+0000 7f4c61ffb640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 0x7f4c5006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.292+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f4c580abfd0 con 0x7f4c641582c0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.293+0000 7f4c695e3640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 0x7f4c5006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.295 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.293+0000 7f4c695e3640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 0x7f4c5006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c014450 tx=0x7f4c5c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.296 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.296+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c640d0210 con 0x7f4c641582c0 2024-04-26T22:28:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.518+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c6406a360 con 0x7f4c641582c0 2024-04-26T22:28:39.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.518+0000 7f4c61ffb640 1 -- 172.21.15.62:0/1138013897 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f4c58079dc0 con 0x7f4c641582c0 2024-04-26T22:28:39.519 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:39.519 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1398555464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1398555464},{"type":"v1","addr":"172.21.15.174:6837","nonce":1398555464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1896156282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1896156282},{"type":"v1","addr":"172.21.15.62:6834","nonce":1896156282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3059612747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3059612747},{"type":"v1","addr":"172.21.15.62:6835","nonce":3059612747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:36.514552+0000","modified":"2024-04-26T22:28:38.513685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6836/4128033051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4128033051},{"type":"v1","addr":"172.21.15.174:6836","nonce":4128033051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5559,"qdb_cluster":[5559]},"id":3}]} 2024-04-26T22:28:39.519 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2024-04-26T22:28:39.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 msgr2=0x7f4c5006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 0x7f4c5006ee40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c014450 tx=0x7f4c5c00b040 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 msgr2=0x7f4c641f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c58000c00 tx=0x7f4c58004730 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 shutdown_connections 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f4c5006c980 0x7f4c5006ee40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4c640cdd80 0x7f4c640ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4c641582c0 0x7f4c641f6050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 --2- 172.21.15.62:0/1138013897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c4c004800 0x7f4c640c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 >> 172.21.15.62:0/1138013897 conn(0x7f4c6409e550 msgr2=0x7f4c640d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 shutdown_connections 2024-04-26T22:28:39.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.520+0000 7f4c6b847640 1 -- 172.21.15.62:0/1138013897 wait complete. 2024-04-26T22:28:39.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T22:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.632+0000 7feaf19b4640 1 Processor -- start 2024-04-26T22:28:39.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.632+0000 7feaf19b4640 1 -- start start 2024-04-26T22:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaf19b4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaf19b4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7feaec160320 0x7feaec0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.633 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaf19b4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaec072e30 con 0x7feaec070130 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaf19b4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7feaec073110 con 0x7feaec160320 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaf19b4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feaec0733f0 con 0x7feaec159a90 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.62:56576/0 (socket says 172.21.15.62:56576) 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feae3fff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7feaec070130 0x7feaec15fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:49468/0 (socket says 172.21.15.62:49468) 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeb7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7feaec160320 0x7feaec0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feae3fff640 1 -- 172.21.15.62:0/3415023791 learned_addr learned my addr 172.21.15.62:0/3415023791 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 -- 172.21.15.62:0/3415023791 >> v2:172.21.15.174:3300/0 conn(0x7feaec160320 msgr2=0x7feaec0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 --2- 172.21.15.62:0/3415023791 >> v2:172.21.15.174:3300/0 conn(0x7feaec160320 0x7feaec0ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 -- 172.21.15.62:0/3415023791 >> v1:172.21.15.174:6789/0 conn(0x7feaec070130 legacy=0x7feaec15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.634 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.633+0000 7feaeaffd640 1 -- 172.21.15.62:0/3415023791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec0736d0 con 0x7feaec159a90 2024-04-26T22:28:39.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.634+0000 7feaeaffd640 1 --2- 172.21.15.62:0/3415023791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feadc0097a0 tx=0x7feadc02c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e9399defe7edba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.634+0000 7feaea7fc640 1 -- 172.21.15.62:0/3415023791 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc039070 con 0x7feaec159a90 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.634+0000 7feaea7fc640 1 -- 172.21.15.62:0/3415023791 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc034440 con 0x7feaec159a90 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.634+0000 7feaea7fc640 1 -- 172.21.15.62:0/3415023791 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc034810 con 0x7feaec159a90 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 -- 172.21.15.62:0/3415023791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 msgr2=0x7feaec06fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3415023791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7feadc0097a0 tx=0x7feadc02c270 comp rx=0 tx=0).stop 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 -- 172.21.15.62:0/3415023791 shutdown_connections 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3415023791 >> v2:172.21.15.174:3300/0 conn(0x7feaec160320 0x7feaec0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3415023791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec06fbf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 -- 172.21.15.62:0/3415023791 >> 172.21.15.62:0/3415023791 conn(0x7feaec0d0d50 msgr2=0x7feaec0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 -- 172.21.15.62:0/3415023791 shutdown_connections 2024-04-26T22:28:39.636 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.635+0000 7feaf19b4640 1 -- 172.21.15.62:0/3415023791 wait complete. 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.636+0000 7feaf19b4640 1 Processor -- start 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.636+0000 7feaf19b4640 1 -- start start 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaec070130 0x7feaec154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feaec159ff0 con 0x7feaec160320 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feaec159e70 con 0x7feaec070130 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaf19b4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feaec0ceef0 con 0x7feaec159a90 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60674/0 (socket says 172.21.15.62:60674) 2024-04-26T22:28:39.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 -- 172.21.15.62:0/3367638717 learned_addr learned my addr 172.21.15.62:0/3367638717 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feae3fff640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeaffd640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaec070130 0x7feaec154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 -- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 msgr2=0x7feaec143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 -- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaec070130 msgr2=0x7feaec154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaec070130 0x7feaec154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 -- 172.21.15.62:0/3367638717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec140de0 con 0x7feaec160320 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.637+0000 7feaeb7fe640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feae4009d30 tx=0x7feae400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.638+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4018020 con 0x7feaec160320 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.638+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae4014480 con 0x7feaec160320 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.638+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec150250 con 0x7feaec160320 2024-04-26T22:28:39.640 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.638+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4014870 con 0x7feaec160320 2024-04-26T22:28:39.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.638+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec150810 con 0x7feaec160320 2024-04-26T22:28:39.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.639+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec06a360 con 0x7feaec160320 2024-04-26T22:28:39.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.639+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7feae4005d10 con 0x7feaec160320 2024-04-26T22:28:39.641 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.640+0000 7feae8ff9640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 0x7feacc06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.640+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7feae408c640 con 0x7feaec160320 2024-04-26T22:28:39.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.640+0000 7feaeaffd640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 0x7feacc06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.642 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.641+0000 7feaeaffd640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 0x7feacc06ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.644 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.644+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feae4022070 con 0x7feaec160320 2024-04-26T22:28:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.828+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaec0d0030 con 0x7feaec160320 2024-04-26T22:28:39.830 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.830+0000 7feae8ff9640 1 -- 172.21.15.62:0/3367638717 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16613 (secure 0 0 0) 0x7feae4010040 con 0x7feaec160320 2024-04-26T22:28:39.830 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:39.830 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":43,"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","created":"2024-04-26T22:26:49.521898+0000","modified":"2024-04-26T22:28:38.142273+0000","last_up_change":"2024-04-26T22:27:00.753481+0000","last_in_change":"2024-04-26T22:26:51.598583+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T22:27:01.653574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T22:28:34.778483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T22:28:35.461326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e0d6ca-021d-48d2-8b0c-c00f30017d21","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6825","nonce":4028840618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6827","nonce":4028840618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6831","nonce":4028840618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":4028840618},{"type":"v1","addr":"172.21.15.62:6829","nonce":4028840618}]},"public_addr":"172.21.15.62:6825/4028840618","cluster_addr":"172.21.15.62:6827/4028840618","heartbeat_back_addr":"172.21.15.62:6831/4028840618","heartbeat_front_addr":"172.21.15.62:6829/4028840618","state":["exists","up"]},{"osd":1,"uuid":"b9753598-6020-4660-980b-cbaf53601dac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6816","nonce":2498719167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6819","nonce":2498719167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6823","nonce":2498719167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":2498719167},{"type":"v1","addr":"172.21.15.62:6821","nonce":2498719167}]},"public_addr":"172.21.15.62:6816/2498719167","cluster_addr":"172.21.15.62:6819/2498719167","heartbeat_back_addr":"172.21.15.62:6823/2498719167","heartbeat_front_addr":"172.21.15.62:6821/2498719167","state":["exists","up"]},{"osd":2,"uuid":"f44b65ac-8e4c-4dc6-9bf4-82b7bd81f643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6801","nonce":1123744749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6803","nonce":1123744749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6807","nonce":1123744749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":1123744749},{"type":"v1","addr":"172.21.15.62:6805","nonce":1123744749}]},"public_addr":"172.21.15.62:6801/1123744749","cluster_addr":"172.21.15.62:6803/1123744749","heartbeat_back_addr":"172.21.15.62:6807/1123744749","heartbeat_front_addr":"172.21.15.62:6805/1123744749","state":["exists","up"]},{"osd":3,"uuid":"4adc6b27-9944-45bb-8f9b-07c3b118cb8c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6809","nonce":3538853986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6811","nonce":3538853986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6815","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6817","nonce":3538853986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":3538853986},{"type":"v1","addr":"172.21.15.62:6813","nonce":3538853986}]},"public_addr":"172.21.15.62:6809/3538853986","cluster_addr":"172.21.15.62:6811/3538853986","heartbeat_back_addr":"172.21.15.62:6817/3538853986","heartbeat_front_addr":"172.21.15.62:6813/3538853986","state":["exists","up"]},{"osd":4,"uuid":"6e454f90-4f3f-48f4-a699-9d309757b2d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6801","nonce":2799507902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6803","nonce":2799507902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6807","nonce":2799507902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2799507902},{"type":"v1","addr":"172.21.15.174:6805","nonce":2799507902}]},"public_addr":"172.21.15.174:6801/2799507902","cluster_addr":"172.21.15.174:6803/2799507902","heartbeat_back_addr":"172.21.15.174:6807/2799507902","heartbeat_front_addr":"172.21.15.174:6805/2799507902","state":["exists","up"]},{"osd":5,"uuid":"bbd7a770-7a75-4e2d-a413-2bff16d3bd05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6817","nonce":3260296682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6819","nonce":3260296682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6823","nonce":3260296682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":3260296682},{"type":"v1","addr":"172.21.15.174:6821","nonce":3260296682}]},"public_addr":"172.21.15.174:6817/3260296682","cluster_addr":"172.21.15.174:6819/3260296682","heartbeat_back_addr":"172.21.15.174:6823/3260296682","heartbeat_front_addr":"172.21.15.174:6821/3260296682","state":["exists","up"]},{"osd":6,"uuid":"dbea0fd7-32c3-4a0c-9963-25e5da3f32da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6825","nonce":341063153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6827","nonce":341063153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6831","nonce":341063153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":341063153},{"type":"v1","addr":"172.21.15.174:6829","nonce":341063153}]},"public_addr":"172.21.15.174:6825/341063153","cluster_addr":"172.21.15.174:6827/341063153","heartbeat_back_addr":"172.21.15.174:6831/341063153","heartbeat_front_addr":"172.21.15.174:6829/341063153","state":["exists","up"]},{"osd":7,"uuid":"b657c762-6714-4b75-bc73-5f19180dd4b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6809","nonce":2710114458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6811","nonce":2710114458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6815","nonce":2710114458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2710114458},{"type":"v1","addr":"172.21.15.174:6813","nonce":2710114458}]},"public_addr":"172.21.15.174:6809/2710114458","cluster_addr":"172.21.15.174:6811/2710114458","heartbeat_back_addr":"172.21.15.174:6815/2710114458","heartbeat_front_addr":"172.21.15.174:6813/2710114458","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.776274+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.773395+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.728814+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.793099+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.765664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.840206+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.769523+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T22:26:56.692960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T22:28:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 msgr2=0x7feacc06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 0x7feacc06ee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7feadc036000 tx=0x7feadc030000 comp rx=0 tx=0).stop 2024-04-26T22:28:39.832 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 msgr2=0x7feaec143e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7feae4009d30 tx=0x7feae400c490 comp rx=0 tx=0).stop 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 shutdown_connections 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7feacc06c980 0x7feacc06ee40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feaec160320 0x7feaec143e80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feaec159a90 0x7feaec143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 --2- 172.21.15.62:0/3367638717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feaec070130 0x7feaec154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 >> 172.21.15.62:0/3367638717 conn(0x7feaec0d0d50 msgr2=0x7feaec158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 shutdown_connections 2024-04-26T22:28:39.833 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.832+0000 7feaf19b4640 1 -- 172.21.15.62:0/3367638717 wait complete. 2024-04-26T22:28:39.843 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T22:28:39.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:39.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.944+0000 7f2c9fb89640 1 Processor -- start 2024-04-26T22:28:39.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.944+0000 7f2c9fb89640 1 -- start start 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9fb89640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c9806ea40 0x7f2c980d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9fb89640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c980cf830 0x7f2c980cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9fb89640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c980b1d70 con 0x7f2c98162e40 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9fb89640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2c98163220 con 0x7f2c9806ea40 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9fb89640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c980d01c0 con 0x7f2c980cf830 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c9806ea40 0x7f2c980d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d925640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f2c98162e40 0x7f2c980b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56606/0 (socket says 172.21.15.62:56606) 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d925640 1 -- 172.21.15.62:0/2179531186 learned_addr learned my addr 172.21.15.62:0/2179531186 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2c9806ea40 0x7f2c980d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60676/0 (socket says 172.21.15.62:60676) 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9e126640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c980cf830 0x7f2c980cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 -- 172.21.15.62:0/2179531186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c980cf830 msgr2=0x7f2c980cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c980cf830 0x7f2c980cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 -- 172.21.15.62:0/2179531186 >> v1:172.21.15.62:6789/0 conn(0x7f2c98162e40 legacy=0x7f2c980b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9c923640 1 -- 172.21.15.62:0/2179531186 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342834217 0 0) 0x7f2c980b1d70 con 0x7f2c98162e40 2024-04-26T22:28:39.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.945+0000 7f2c9d124640 1 -- 172.21.15.62:0/2179531186 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c980d1960 con 0x7f2c9806ea40 2024-04-26T22:28:39.947 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9d124640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9806ea40 0x7f2c980d1340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ae2ac4bc3e9384 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9c923640 1 -- 172.21.15.62:0/2179531186 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88037070 con 0x7f2c9806ea40 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9c923640 1 -- 172.21.15.62:0/2179531186 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8802ae50 con 0x7f2c9806ea40 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9c923640 1 -- 172.21.15.62:0/2179531186 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88032660 con 0x7f2c9806ea40 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 -- 172.21.15.62:0/2179531186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9806ea40 msgr2=0x7f2c980d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9806ea40 0x7f2c980d1340 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802a100 comp rx=0 tx=0).stop 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 -- 172.21.15.62:0/2179531186 shutdown_connections 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c9806ea40 0x7f2c980d1340 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/2179531186 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c980cf830 0x7f2c980cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.946+0000 7f2c9fb89640 1 -- 172.21.15.62:0/2179531186 >> 172.21.15.62:0/2179531186 conn(0x7f2c9809ba50 msgr2=0x7f2c9809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.947+0000 7f2c9fb89640 1 -- 172.21.15.62:0/2179531186 shutdown_connections 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.947+0000 7f2c9fb89640 1 -- 172.21.15.62:0/2179531186 wait complete. 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 Processor -- start 2024-04-26T22:28:39.948 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 -- start start 2024-04-26T22:28:39.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c9806ea40 0x7f2c980b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.950 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c980cf830 0x7f2c980c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980d1f20 con 0x7f2c98162e40 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980d0eb0 con 0x7f2c980cf830 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.948+0000 7f2c9fb89640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c980ce010 con 0x7f2c9806ea40 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60684/0 (socket says 172.21.15.62:60684) 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 -- 172.21.15.62:0/1336347572 learned_addr learned my addr 172.21.15.62:0/1336347572 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9d925640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c9806ea40 0x7f2c980b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9d124640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c980cf830 0x7f2c980c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 -- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c9806ea40 msgr2=0x7f2c980b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c9806ea40 0x7f2c980b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 -- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c980cf830 msgr2=0x7f2c980c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c980cf830 0x7f2c980c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:39.951 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 -- 172.21.15.62:0/1336347572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c980c8e40 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9e126640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2c90007e70 tx=0x7f2c9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90014620 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c90019070 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c980b5c30 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90014a20 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.949+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c980b61f0 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.950+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c98068db0 con 0x7f2c98162e40 2024-04-26T22:28:39.952 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.951+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2c90014bc0 con 0x7f2c98162e40 2024-04-26T22:28:39.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.951+0000 7f2c8e7fc640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 0x7f2c7c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:39.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.951+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2c9008c350 con 0x7f2c98162e40 2024-04-26T22:28:39.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.951+0000 7f2c9d925640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 0x7f2c7c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:39.953 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.952+0000 7f2c9d925640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 0x7f2c7c06ee70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c94007fd0 tx=0x7f2c94007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:39.955 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:39.955+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c98068db0 con 0x7f2c98162e40 2024-04-26T22:28:40.135 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.135+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c9806cd70 con 0x7f2c7c06c9b0 2024-04-26T22:28:40.142 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.142+0000 7f2c8e7fc640 1 -- 172.21.15.62:0/1336347572 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7f2c9806cd70 con 0x7f2c7c06c9b0 2024-04-26T22:28:40.144 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:40.146 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 msgr2=0x7f2c7c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 0x7f2c7c06ee70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2c94007fd0 tx=0x7f2c94007de0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 msgr2=0x7f2c981533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2c90007e70 tx=0x7f2c9000e4b0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 shutdown_connections 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2c7c06c9b0 0x7f2c7c06ee70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2c98162e40 0x7f2c981533e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c980cf830 0x7f2c980c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 --2- 172.21.15.62:0/1336347572 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c9806ea40 0x7f2c980b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 >> 172.21.15.62:0/1336347572 conn(0x7f2c9809ba50 msgr2=0x7f2c98071660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 shutdown_connections 2024-04-26T22:28:40.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.145+0000 7f2c9fb89640 1 -- 172.21.15.62:0/1336347572 wait complete. 2024-04-26T22:28:40.156 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-26T22:28:39.658448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221032,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753688,"statfs":{"total":773094113280,"available":772867776512,"internally_reserved":0,"allocated":4972544,"data_stored":2676652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.524630"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148975+0000","last_change":"2024-04-26T22:28:36.148975+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148975+0000","last_undegraded":"2024-04-26T22:28:36.148975+0000","last_fullsized":"2024-04-26T22:28:36.148975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148634+0000","last_change":"2024-04-26T22:28:36.148634+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148634+0000","last_undegraded":"2024-04-26T22:28:36.148634+0000","last_fullsized":"2024-04-26T22:28:36.148634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157768+0000","last_change":"2024-04-26T22:28:36.157768+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157768+0000","last_undegraded":"2024-04-26T22:28:36.157768+0000","last_fullsized":"2024-04-26T22:28:36.157768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157943+0000","last_change":"2024-04-26T22:28:36.157943+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157943+0000","last_undegraded":"2024-04-26T22:28:36.157943+0000","last_fullsized":"2024-04-26T22:28:36.157943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148758+0000","last_change":"2024-04-26T22:28:36.148758+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148758+0000","last_undegraded":"2024-04-26T22:28:36.148758+0000","last_fullsized":"2024-04-26T22:28:36.148758+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151939+0000","last_change":"2024-04-26T22:28:36.151939+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151939+0000","last_undegraded":"2024-04-26T22:28:36.151939+0000","last_fullsized":"2024-04-26T22:28:36.151939+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150312+0000","last_change":"2024-04-26T22:28:36.150312+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150312+0000","last_undegraded":"2024-04-26T22:28:36.150312+0000","last_fullsized":"2024-04-26T22:28:36.150312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157416+0000","last_change":"2024-04-26T22:28:36.157416+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157416+0000","last_undegraded":"2024-04-26T22:28:36.157416+0000","last_fullsized":"2024-04-26T22:28:36.157416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157243+0000","last_change":"2024-04-26T22:28:36.157243+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157243+0000","last_undegraded":"2024-04-26T22:28:36.157243+0000","last_fullsized":"2024-04-26T22:28:36.157243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156453+0000","last_change":"2024-04-26T22:28:36.156453+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156453+0000","last_undegraded":"2024-04-26T22:28:36.156453+0000","last_fullsized":"2024-04-26T22:28:36.156453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154387+0000","last_change":"2024-04-26T22:28:36.154387+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154387+0000","last_undegraded":"2024-04-26T22:28:36.154387+0000","last_fullsized":"2024-04-26T22:28:36.154387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150951+0000","last_change":"2024-04-26T22:28:36.150951+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150951+0000","last_undegraded":"2024-04-26T22:28:36.150951+0000","last_fullsized":"2024-04-26T22:28:36.150951+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147261+0000","last_change":"2024-04-26T22:28:36.147261+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147261+0000","last_undegraded":"2024-04-26T22:28:36.147261+0000","last_fullsized":"2024-04-26T22:28:36.147261+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154697+0000","last_change":"2024-04-26T22:28:36.154697+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154697+0000","last_undegraded":"2024-04-26T22:28:36.154697+0000","last_fullsized":"2024-04-26T22:28:36.154697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152160+0000","last_change":"2024-04-26T22:28:36.152160+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152160+0000","last_undegraded":"2024-04-26T22:28:36.152160+0000","last_fullsized":"2024-04-26T22:28:36.152160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151319+0000","last_change":"2024-04-26T22:28:36.151319+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151319+0000","last_undegraded":"2024-04-26T22:28:36.151319+0000","last_fullsized":"2024-04-26T22:28:36.151319+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157224+0000","last_change":"2024-04-26T22:28:36.157224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157224+0000","last_undegraded":"2024-04-26T22:28:36.157224+0000","last_fullsized":"2024-04-26T22:28:36.157224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.159081+0000","last_change":"2024-04-26T22:28:36.159081+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.159081+0000","last_undegraded":"2024-04-26T22:28:36.159081+0000","last_fullsized":"2024-04-26T22:28:36.159081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152582+0000","last_change":"2024-04-26T22:28:36.152582+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152582+0000","last_undegraded":"2024-04-26T22:28:36.152582+0000","last_fullsized":"2024-04-26T22:28:36.152582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148009+0000","last_change":"2024-04-26T22:28:36.148009+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148009+0000","last_undegraded":"2024-04-26T22:28:36.148009+0000","last_fullsized":"2024-04-26T22:28:36.148009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150983+0000","last_change":"2024-04-26T22:28:36.150983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150983+0000","last_undegraded":"2024-04-26T22:28:36.150983+0000","last_fullsized":"2024-04-26T22:28:36.150983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154224+0000","last_change":"2024-04-26T22:28:36.154224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154224+0000","last_undegraded":"2024-04-26T22:28:36.154224+0000","last_fullsized":"2024-04-26T22:28:36.154224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152240+0000","last_change":"2024-04-26T22:28:36.152240+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152240+0000","last_undegraded":"2024-04-26T22:28:36.152240+0000","last_fullsized":"2024-04-26T22:28:36.152240+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150753+0000","last_change":"2024-04-26T22:28:36.150753+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150753+0000","last_undegraded":"2024-04-26T22:28:36.150753+0000","last_fullsized":"2024-04-26T22:28:36.150753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154621+0000","last_change":"2024-04-26T22:28:36.154621+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154621+0000","last_undegraded":"2024-04-26T22:28:36.154621+0000","last_fullsized":"2024-04-26T22:28:36.154621+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157292+0000","last_change":"2024-04-26T22:28:36.157292+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157292+0000","last_undegraded":"2024-04-26T22:28:36.157292+0000","last_fullsized":"2024-04-26T22:28:36.157292+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150520+0000","last_change":"2024-04-26T22:28:36.150520+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150520+0000","last_undegraded":"2024-04-26T22:28:36.150520+0000","last_fullsized":"2024-04-26T22:28:36.150520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151361+0000","last_change":"2024-04-26T22:28:36.151361+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151361+0000","last_undegraded":"2024-04-26T22:28:36.151361+0000","last_fullsized":"2024-04-26T22:28:36.151361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158720+0000","last_change":"2024-04-26T22:28:36.158720+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158720+0000","last_undegraded":"2024-04-26T22:28:36.158720+0000","last_fullsized":"2024-04-26T22:28:36.158720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150207+0000","last_change":"2024-04-26T22:28:36.150207+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150207+0000","last_undegraded":"2024-04-26T22:28:36.150207+0000","last_fullsized":"2024-04-26T22:28:36.150207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147125+0000","last_change":"2024-04-26T22:28:36.147125+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147125+0000","last_undegraded":"2024-04-26T22:28:36.147125+0000","last_fullsized":"2024-04-26T22:28:36.147125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159896+0000","last_change":"2024-04-26T22:28:36.157128+0000","last_active":"2024-04-26T22:28:36.159896+0000","last_peered":"2024-04-26T22:28:36.159896+0000","last_clean":"2024-04-26T22:28:36.159896+0000","last_became_active":"2024-04-26T22:28:36.156854+0000","last_became_peered":"2024-04-26T22:28:36.156854+0000","last_unstale":"2024-04-26T22:28:36.159896+0000","last_undegraded":"2024-04-26T22:28:36.159896+0000","last_fullsized":"2024-04-26T22:28:36.159896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154011+0000","last_change":"2024-04-26T22:28:36.154011+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154011+0000","last_undegraded":"2024-04-26T22:28:36.154011+0000","last_fullsized":"2024-04-26T22:28:36.154011+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170598+0000","last_change":"2024-04-26T22:28:36.165408+0000","last_active":"2024-04-26T22:28:36.170598+0000","last_peered":"2024-04-26T22:28:36.170598+0000","last_clean":"2024-04-26T22:28:36.170598+0000","last_became_active":"2024-04-26T22:28:36.165110+0000","last_became_peered":"2024-04-26T22:28:36.165110+0000","last_unstale":"2024-04-26T22:28:36.170598+0000","last_undegraded":"2024-04-26T22:28:36.170598+0000","last_fullsized":"2024-04-26T22:28:36.170598+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148308+0000","last_change":"2024-04-26T22:28:36.148308+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148308+0000","last_undegraded":"2024-04-26T22:28:36.148308+0000","last_fullsized":"2024-04-26T22:28:36.148308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.156144+0000","last_change":"2024-04-26T22:28:36.154806+0000","last_active":"2024-04-26T22:28:36.156144+0000","last_peered":"2024-04-26T22:28:36.156144+0000","last_clean":"2024-04-26T22:28:36.156144+0000","last_became_active":"2024-04-26T22:28:36.154497+0000","last_became_peered":"2024-04-26T22:28:36.154497+0000","last_unstale":"2024-04-26T22:28:36.156144+0000","last_undegraded":"2024-04-26T22:28:36.156144+0000","last_fullsized":"2024-04-26T22:28:36.156144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148250+0000","last_change":"2024-04-26T22:28:36.148250+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148250+0000","last_undegraded":"2024-04-26T22:28:36.148250+0000","last_fullsized":"2024-04-26T22:28:36.148250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170452+0000","last_change":"2024-04-26T22:28:36.165313+0000","last_active":"2024-04-26T22:28:36.170452+0000","last_peered":"2024-04-26T22:28:36.170452+0000","last_clean":"2024-04-26T22:28:36.170452+0000","last_became_active":"2024-04-26T22:28:36.165008+0000","last_became_peered":"2024-04-26T22:28:36.165008+0000","last_unstale":"2024-04-26T22:28:36.170452+0000","last_undegraded":"2024-04-26T22:28:36.170452+0000","last_fullsized":"2024-04-26T22:28:36.170452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:56.424170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157367+0000","last_change":"2024-04-26T22:28:36.157367+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157367+0000","last_undegraded":"2024-04-26T22:28:36.157367+0000","last_fullsized":"2024-04-26T22:28:36.157367+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159950+0000","last_change":"2024-04-26T22:28:36.157297+0000","last_active":"2024-04-26T22:28:36.159950+0000","last_peered":"2024-04-26T22:28:36.159950+0000","last_clean":"2024-04-26T22:28:36.159950+0000","last_became_active":"2024-04-26T22:28:36.157002+0000","last_became_peered":"2024-04-26T22:28:36.157002+0000","last_unstale":"2024-04-26T22:28:36.159950+0000","last_undegraded":"2024-04-26T22:28:36.159950+0000","last_fullsized":"2024-04-26T22:28:36.159950+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:36.488188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155983+0000","last_change":"2024-04-26T22:28:36.155983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155983+0000","last_undegraded":"2024-04-26T22:28:36.155983+0000","last_fullsized":"2024-04-26T22:28:36.155983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.172286+0000","last_change":"2024-04-26T22:28:36.167783+0000","last_active":"2024-04-26T22:28:36.172286+0000","last_peered":"2024-04-26T22:28:36.172286+0000","last_clean":"2024-04-26T22:28:36.172286+0000","last_became_active":"2024-04-26T22:28:36.167480+0000","last_became_peered":"2024-04-26T22:28:36.167480+0000","last_unstale":"2024-04-26T22:28:36.172286+0000","last_undegraded":"2024-04-26T22:28:36.172286+0000","last_fullsized":"2024-04-26T22:28:36.172286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152072+0000","last_change":"2024-04-26T22:28:36.152072+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152072+0000","last_undegraded":"2024-04-26T22:28:36.152072+0000","last_fullsized":"2024-04-26T22:28:36.152072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167154+0000","last_change":"2024-04-26T22:28:36.161210+0000","last_active":"2024-04-26T22:28:36.167154+0000","last_peered":"2024-04-26T22:28:36.167154+0000","last_clean":"2024-04-26T22:28:36.167154+0000","last_became_active":"2024-04-26T22:28:36.160683+0000","last_became_peered":"2024-04-26T22:28:36.160683+0000","last_unstale":"2024-04-26T22:28:36.167154+0000","last_undegraded":"2024-04-26T22:28:36.167154+0000","last_fullsized":"2024-04-26T22:28:36.167154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153574+0000","last_change":"2024-04-26T22:28:36.153574+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153574+0000","last_undegraded":"2024-04-26T22:28:36.153574+0000","last_fullsized":"2024-04-26T22:28:36.153574+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164527+0000","last_change":"2024-04-26T22:28:36.162766+0000","last_active":"2024-04-26T22:28:36.164527+0000","last_peered":"2024-04-26T22:28:36.164527+0000","last_clean":"2024-04-26T22:28:36.164527+0000","last_became_active":"2024-04-26T22:28:36.162468+0000","last_became_peered":"2024-04-26T22:28:36.162468+0000","last_unstale":"2024-04-26T22:28:36.164527+0000","last_undegraded":"2024-04-26T22:28:36.164527+0000","last_fullsized":"2024-04-26T22:28:36.164527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158533+0000","last_change":"2024-04-26T22:28:36.158533+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158533+0000","last_undegraded":"2024-04-26T22:28:36.158533+0000","last_fullsized":"2024-04-26T22:28:36.158533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164922+0000","last_change":"2024-04-26T22:28:36.162336+0000","last_active":"2024-04-26T22:28:36.164922+0000","last_peered":"2024-04-26T22:28:36.164922+0000","last_clean":"2024-04-26T22:28:36.164922+0000","last_became_active":"2024-04-26T22:28:36.162067+0000","last_became_peered":"2024-04-26T22:28:36.162067+0000","last_unstale":"2024-04-26T22:28:36.164922+0000","last_undegraded":"2024-04-26T22:28:36.164922+0000","last_fullsized":"2024-04-26T22:28:36.164922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148121+0000","last_change":"2024-04-26T22:28:36.148121+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148121+0000","last_undegraded":"2024-04-26T22:28:36.148121+0000","last_fullsized":"2024-04-26T22:28:36.148121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164607+0000","last_change":"2024-04-26T22:28:36.161738+0000","last_active":"2024-04-26T22:28:36.164607+0000","last_peered":"2024-04-26T22:28:36.164607+0000","last_clean":"2024-04-26T22:28:36.164607+0000","last_became_active":"2024-04-26T22:28:36.161489+0000","last_became_peered":"2024-04-26T22:28:36.161489+0000","last_unstale":"2024-04-26T22:28:36.164607+0000","last_undegraded":"2024-04-26T22:28:36.164607+0000","last_fullsized":"2024-04-26T22:28:36.164607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150196+0000","last_change":"2024-04-26T22:28:36.150196+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150196+0000","last_undegraded":"2024-04-26T22:28:36.150196+0000","last_fullsized":"2024-04-26T22:28:36.150196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164669+0000","last_change":"2024-04-26T22:28:36.159715+0000","last_active":"2024-04-26T22:28:36.164669+0000","last_peered":"2024-04-26T22:28:36.164669+0000","last_clean":"2024-04-26T22:28:36.164669+0000","last_became_active":"2024-04-26T22:28:36.159437+0000","last_became_peered":"2024-04-26T22:28:36.159437+0000","last_unstale":"2024-04-26T22:28:36.164669+0000","last_undegraded":"2024-04-26T22:28:36.164669+0000","last_fullsized":"2024-04-26T22:28:36.164669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149529+0000","last_change":"2024-04-26T22:28:36.149529+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149529+0000","last_undegraded":"2024-04-26T22:28:36.149529+0000","last_fullsized":"2024-04-26T22:28:36.149529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165270+0000","last_change":"2024-04-26T22:28:36.163369+0000","last_active":"2024-04-26T22:28:36.165270+0000","last_peered":"2024-04-26T22:28:36.165270+0000","last_clean":"2024-04-26T22:28:36.165270+0000","last_became_active":"2024-04-26T22:28:36.163060+0000","last_became_peered":"2024-04-26T22:28:36.163060+0000","last_unstale":"2024-04-26T22:28:36.165270+0000","last_undegraded":"2024-04-26T22:28:36.165270+0000","last_fullsized":"2024-04-26T22:28:36.165270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:58.196234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151707+0000","last_change":"2024-04-26T22:28:36.151707+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151707+0000","last_undegraded":"2024-04-26T22:28:36.151707+0000","last_fullsized":"2024-04-26T22:28:36.151707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.166646+0000","last_change":"2024-04-26T22:28:36.165081+0000","last_active":"2024-04-26T22:28:36.166646+0000","last_peered":"2024-04-26T22:28:36.166646+0000","last_clean":"2024-04-26T22:28:36.166646+0000","last_became_active":"2024-04-26T22:28:36.164792+0000","last_became_peered":"2024-04-26T22:28:36.164792+0000","last_unstale":"2024-04-26T22:28:36.166646+0000","last_undegraded":"2024-04-26T22:28:36.166646+0000","last_fullsized":"2024-04-26T22:28:36.166646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:54.680123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155483+0000","last_change":"2024-04-26T22:28:36.155483+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155483+0000","last_undegraded":"2024-04-26T22:28:36.155483+0000","last_fullsized":"2024-04-26T22:28:36.155483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162204+0000","last_change":"2024-04-26T22:28:36.158147+0000","last_active":"2024-04-26T22:28:36.162204+0000","last_peered":"2024-04-26T22:28:36.162204+0000","last_clean":"2024-04-26T22:28:36.162204+0000","last_became_active":"2024-04-26T22:28:36.157890+0000","last_became_peered":"2024-04-26T22:28:36.157890+0000","last_unstale":"2024-04-26T22:28:36.162204+0000","last_undegraded":"2024-04-26T22:28:36.162204+0000","last_fullsized":"2024-04-26T22:28:36.162204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151703+0000","last_change":"2024-04-26T22:28:36.151703+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151703+0000","last_undegraded":"2024-04-26T22:28:36.151703+0000","last_fullsized":"2024-04-26T22:28:36.151703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165776+0000","last_change":"2024-04-26T22:28:36.164416+0000","last_active":"2024-04-26T22:28:36.165776+0000","last_peered":"2024-04-26T22:28:36.165776+0000","last_clean":"2024-04-26T22:28:36.165776+0000","last_became_active":"2024-04-26T22:28:36.164074+0000","last_became_peered":"2024-04-26T22:28:36.164074+0000","last_unstale":"2024-04-26T22:28:36.165776+0000","last_undegraded":"2024-04-26T22:28:36.165776+0000","last_fullsized":"2024-04-26T22:28:36.165776+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:44.778533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148440+0000","last_change":"2024-04-26T22:28:36.148440+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148440+0000","last_undegraded":"2024-04-26T22:28:36.148440+0000","last_fullsized":"2024-04-26T22:28:36.148440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.168241+0000","last_change":"2024-04-26T22:28:36.167663+0000","last_active":"2024-04-26T22:28:36.168241+0000","last_peered":"2024-04-26T22:28:36.168241+0000","last_clean":"2024-04-26T22:28:36.168241+0000","last_became_active":"2024-04-26T22:28:36.167430+0000","last_became_peered":"2024-04-26T22:28:36.167430+0000","last_unstale":"2024-04-26T22:28:36.168241+0000","last_undegraded":"2024-04-26T22:28:36.168241+0000","last_fullsized":"2024-04-26T22:28:36.168241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146555+0000","last_change":"2024-04-26T22:28:36.146555+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146555+0000","last_undegraded":"2024-04-26T22:28:36.146555+0000","last_fullsized":"2024-04-26T22:28:36.146555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155478+0000","last_change":"2024-04-26T22:28:36.152997+0000","last_active":"2024-04-26T22:28:36.155478+0000","last_peered":"2024-04-26T22:28:36.155478+0000","last_clean":"2024-04-26T22:28:36.155478+0000","last_became_active":"2024-04-26T22:28:36.152694+0000","last_became_peered":"2024-04-26T22:28:36.152694+0000","last_unstale":"2024-04-26T22:28:36.155478+0000","last_undegraded":"2024-04-26T22:28:36.155478+0000","last_fullsized":"2024-04-26T22:28:36.155478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:42.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153502+0000","last_change":"2024-04-26T22:28:36.153502+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153502+0000","last_undegraded":"2024-04-26T22:28:36.153502+0000","last_fullsized":"2024-04-26T22:28:36.153502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164982+0000","last_change":"2024-04-26T22:28:36.162940+0000","last_active":"2024-04-26T22:28:36.164982+0000","last_peered":"2024-04-26T22:28:36.164982+0000","last_clean":"2024-04-26T22:28:36.164982+0000","last_became_active":"2024-04-26T22:28:36.162676+0000","last_became_peered":"2024-04-26T22:28:36.162676+0000","last_unstale":"2024-04-26T22:28:36.164982+0000","last_undegraded":"2024-04-26T22:28:36.164982+0000","last_fullsized":"2024-04-26T22:28:36.164982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153988+0000","last_change":"2024-04-26T22:28:36.153988+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153988+0000","last_undegraded":"2024-04-26T22:28:36.153988+0000","last_fullsized":"2024-04-26T22:28:36.153988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167075+0000","last_change":"2024-04-26T22:28:36.157167+0000","last_active":"2024-04-26T22:28:36.167075+0000","last_peered":"2024-04-26T22:28:36.167075+0000","last_clean":"2024-04-26T22:28:36.167075+0000","last_became_active":"2024-04-26T22:28:36.156867+0000","last_became_peered":"2024-04-26T22:28:36.156867+0000","last_unstale":"2024-04-26T22:28:36.167075+0000","last_undegraded":"2024-04-26T22:28:36.167075+0000","last_fullsized":"2024-04-26T22:28:36.167075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154215+0000","last_change":"2024-04-26T22:28:36.154215+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154215+0000","last_undegraded":"2024-04-26T22:28:36.154215+0000","last_fullsized":"2024-04-26T22:28:36.154215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.163612+0000","last_change":"2024-04-26T22:28:36.160489+0000","last_active":"2024-04-26T22:28:36.163612+0000","last_peered":"2024-04-26T22:28:36.163612+0000","last_clean":"2024-04-26T22:28:36.163612+0000","last_became_active":"2024-04-26T22:28:36.160227+0000","last_became_peered":"2024-04-26T22:28:36.160227+0000","last_unstale":"2024-04-26T22:28:36.163612+0000","last_undegraded":"2024-04-26T22:28:36.163612+0000","last_fullsized":"2024-04-26T22:28:36.163612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:23.581702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149391+0000","last_change":"2024-04-26T22:28:36.149391+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149391+0000","last_undegraded":"2024-04-26T22:28:36.149391+0000","last_fullsized":"2024-04-26T22:28:36.149391+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159929+0000","last_change":"2024-04-26T22:28:36.156431+0000","last_active":"2024-04-26T22:28:36.159929+0000","last_peered":"2024-04-26T22:28:36.159929+0000","last_clean":"2024-04-26T22:28:36.159929+0000","last_became_active":"2024-04-26T22:28:36.156129+0000","last_became_peered":"2024-04-26T22:28:36.156129+0000","last_unstale":"2024-04-26T22:28:36.159929+0000","last_undegraded":"2024-04-26T22:28:36.159929+0000","last_fullsized":"2024-04-26T22:28:36.159929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:07.246183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146638+0000","last_change":"2024-04-26T22:28:36.146638+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146638+0000","last_undegraded":"2024-04-26T22:28:36.146638+0000","last_fullsized":"2024-04-26T22:28:36.146638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167053+0000","last_change":"2024-04-26T22:28:36.156716+0000","last_active":"2024-04-26T22:28:36.167053+0000","last_peered":"2024-04-26T22:28:36.167053+0000","last_clean":"2024-04-26T22:28:36.167053+0000","last_became_active":"2024-04-26T22:28:36.156424+0000","last_became_peered":"2024-04-26T22:28:36.156424+0000","last_unstale":"2024-04-26T22:28:36.167053+0000","last_undegraded":"2024-04-26T22:28:36.167053+0000","last_fullsized":"2024-04-26T22:28:36.167053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146530+0000","last_change":"2024-04-26T22:28:36.146530+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146530+0000","last_undegraded":"2024-04-26T22:28:36.146530+0000","last_fullsized":"2024-04-26T22:28:36.146530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.160043+0000","last_change":"2024-04-26T22:28:36.157532+0000","last_active":"2024-04-26T22:28:36.160043+0000","last_peered":"2024-04-26T22:28:36.160043+0000","last_clean":"2024-04-26T22:28:36.160043+0000","last_became_active":"2024-04-26T22:28:36.157238+0000","last_became_peered":"2024-04-26T22:28:36.157238+0000","last_unstale":"2024-04-26T22:28:36.160043+0000","last_undegraded":"2024-04-26T22:28:36.160043+0000","last_fullsized":"2024-04-26T22:28:36.160043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:16.612120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151105+0000","last_change":"2024-04-26T22:28:36.151105+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151105+0000","last_undegraded":"2024-04-26T22:28:36.151105+0000","last_fullsized":"2024-04-26T22:28:36.151105+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162295+0000","last_change":"2024-04-26T22:28:36.157860+0000","last_active":"2024-04-26T22:28:36.162295+0000","last_peered":"2024-04-26T22:28:36.162295+0000","last_clean":"2024-04-26T22:28:36.162295+0000","last_became_active":"2024-04-26T22:28:36.157588+0000","last_became_peered":"2024-04-26T22:28:36.157588+0000","last_unstale":"2024-04-26T22:28:36.162295+0000","last_undegraded":"2024-04-26T22:28:36.162295+0000","last_fullsized":"2024-04-26T22:28:36.162295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153545+0000","last_change":"2024-04-26T22:28:36.153545+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153545+0000","last_undegraded":"2024-04-26T22:28:36.153545+0000","last_fullsized":"2024-04-26T22:28:36.153545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167355+0000","last_change":"2024-04-26T22:28:36.165811+0000","last_active":"2024-04-26T22:28:36.167355+0000","last_peered":"2024-04-26T22:28:36.167355+0000","last_clean":"2024-04-26T22:28:36.167355+0000","last_became_active":"2024-04-26T22:28:36.165461+0000","last_became_peered":"2024-04-26T22:28:36.165461+0000","last_unstale":"2024-04-26T22:28:36.167355+0000","last_undegraded":"2024-04-26T22:28:36.167355+0000","last_fullsized":"2024-04-26T22:28:36.167355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:18.602353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153458+0000","last_change":"2024-04-26T22:28:36.153458+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153458+0000","last_undegraded":"2024-04-26T22:28:36.153458+0000","last_fullsized":"2024-04-26T22:28:36.153458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164692+0000","last_change":"2024-04-26T22:28:36.159775+0000","last_active":"2024-04-26T22:28:36.164692+0000","last_peered":"2024-04-26T22:28:36.164692+0000","last_clean":"2024-04-26T22:28:36.164692+0000","last_became_active":"2024-04-26T22:28:36.159490+0000","last_became_peered":"2024-04-26T22:28:36.159490+0000","last_unstale":"2024-04-26T22:28:36.164692+0000","last_undegraded":"2024-04-26T22:28:36.164692+0000","last_fullsized":"2024-04-26T22:28:36.164692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149079+0000","last_change":"2024-04-26T22:28:36.149079+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149079+0000","last_undegraded":"2024-04-26T22:28:36.149079+0000","last_fullsized":"2024-04-26T22:28:36.149079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155941+0000","last_change":"2024-04-26T22:28:36.154743+0000","last_active":"2024-04-26T22:28:36.155941+0000","last_peered":"2024-04-26T22:28:36.155941+0000","last_clean":"2024-04-26T22:28:36.155941+0000","last_became_active":"2024-04-26T22:28:36.154433+0000","last_became_peered":"2024-04-26T22:28:36.154433+0000","last_unstale":"2024-04-26T22:28:36.155941+0000","last_undegraded":"2024-04-26T22:28:36.155941+0000","last_fullsized":"2024-04-26T22:28:36.155941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146336+0000","last_change":"2024-04-26T22:28:36.146336+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146336+0000","last_undegraded":"2024-04-26T22:28:36.146336+0000","last_fullsized":"2024-04-26T22:28:36.146336+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164732+0000","last_change":"2024-04-26T22:28:36.160465+0000","last_active":"2024-04-26T22:28:36.164732+0000","last_peered":"2024-04-26T22:28:36.164732+0000","last_clean":"2024-04-26T22:28:36.164732+0000","last_became_active":"2024-04-26T22:28:36.160206+0000","last_became_peered":"2024-04-26T22:28:36.160206+0000","last_unstale":"2024-04-26T22:28:36.164732+0000","last_undegraded":"2024-04-26T22:28:36.164732+0000","last_fullsized":"2024-04-26T22:28:36.164732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:47.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146383+0000","last_change":"2024-04-26T22:28:36.146383+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146383+0000","last_undegraded":"2024-04-26T22:28:36.146383+0000","last_fullsized":"2024-04-26T22:28:36.146383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164966+0000","last_change":"2024-04-26T22:28:36.163006+0000","last_active":"2024-04-26T22:28:36.164966+0000","last_peered":"2024-04-26T22:28:36.164966+0000","last_clean":"2024-04-26T22:28:36.164966+0000","last_became_active":"2024-04-26T22:28:36.162735+0000","last_became_peered":"2024-04-26T22:28:36.162735+0000","last_unstale":"2024-04-26T22:28:36.164966+0000","last_undegraded":"2024-04-26T22:28:36.164966+0000","last_fullsized":"2024-04-26T22:28:36.164966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147713+0000","last_change":"2024-04-26T22:28:36.147713+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147713+0000","last_undegraded":"2024-04-26T22:28:36.147713+0000","last_fullsized":"2024-04-26T22:28:36.147713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.161202+0000","last_change":"2024-04-26T22:28:36.159382+0000","last_active":"2024-04-26T22:28:36.161202+0000","last_peered":"2024-04-26T22:28:36.161202+0000","last_clean":"2024-04-26T22:28:36.161202+0000","last_became_active":"2024-04-26T22:28:36.159016+0000","last_became_peered":"2024-04-26T22:28:36.159016+0000","last_unstale":"2024-04-26T22:28:36.161202+0000","last_undegraded":"2024-04-26T22:28:36.161202+0000","last_fullsized":"2024-04-26T22:28:36.161202+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156626+0000","last_change":"2024-04-26T22:28:36.156626+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156626+0000","last_undegraded":"2024-04-26T22:28:36.156626+0000","last_fullsized":"2024-04-26T22:28:36.156626+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164507+0000","last_change":"2024-04-26T22:28:36.162470+0000","last_active":"2024-04-26T22:28:36.164507+0000","last_peered":"2024-04-26T22:28:36.164507+0000","last_clean":"2024-04-26T22:28:36.164507+0000","last_became_active":"2024-04-26T22:28:36.162164+0000","last_became_peered":"2024-04-26T22:28:36.162164+0000","last_unstale":"2024-04-26T22:28:36.164507+0000","last_undegraded":"2024-04-26T22:28:36.164507+0000","last_fullsized":"2024-04-26T22:28:36.164507+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155799+0000","last_change":"2024-04-26T22:28:36.155799+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155799+0000","last_undegraded":"2024-04-26T22:28:36.155799+0000","last_fullsized":"2024-04-26T22:28:36.155799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167245+0000","last_change":"2024-04-26T22:28:36.162279+0000","last_active":"2024-04-26T22:28:36.167245+0000","last_peered":"2024-04-26T22:28:36.167245+0000","last_clean":"2024-04-26T22:28:36.167245+0000","last_became_active":"2024-04-26T22:28:36.161730+0000","last_became_peered":"2024-04-26T22:28:36.161730+0000","last_unstale":"2024-04-26T22:28:36.167245+0000","last_undegraded":"2024-04-26T22:28:36.167245+0000","last_fullsized":"2024-04-26T22:28:36.167245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.152298+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:36.152298+0000","last_peered":"2024-04-26T22:28:36.152298+0000","last_clean":"2024-04-26T22:28:36.152298+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:36.152298+0000","last_undegraded":"2024-04-26T22:28:36.152298+0000","last_fullsized":"2024-04-26T22:28:36.152298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":774224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:40.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 Processor -- start 2024-04-26T22:28:40.297 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 -- start start 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f208c15f760 0x7f208c0b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208c0d3de0 con 0x7f208c0d22e0 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208c0d40c0 con 0x7f208c15f380 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.297+0000 7f2092485640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f208c0d43a0 con 0x7f208c15f760 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f208b7fe640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f208c0d22e0 0x7f208c0b8a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56614/0 (socket says 172.21.15.62:56614) 2024-04-26T22:28:40.298 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f208b7fe640 1 -- 172.21.15.62:0/2167426741 learned_addr learned my addr 172.21.15.62:0/2167426741 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f2090a22640 1 --2- 172.21.15.62:0/2167426741 >> v2:172.21.15.62:3300/0 conn(0x7f208c15f760 0x7f208c0b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f2090a22640 1 -- 172.21.15.62:0/2167426741 >> v1:172.21.15.174:6789/0 conn(0x7f208c15f380 legacy=0x7f208c0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f2090a22640 1 -- 172.21.15.62:0/2167426741 >> v1:172.21.15.62:6789/0 conn(0x7f208c0d22e0 legacy=0x7f208c0b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f2090a22640 1 -- 172.21.15.62:0/2167426741 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c0b8280 con 0x7f208c15f760 2024-04-26T22:28:40.299 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.298+0000 7f208affd640 1 -- 172.21.15.62:0/2167426741 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420699144 0 0) 0x7f208c0d3de0 con 0x7f208c0d22e0 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2090a22640 1 --2- 172.21.15.62:0/2167426741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c15f760 0x7f208c0b9190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f208c0d4520 tx=0x7f208402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d918360025580a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f208affd640 1 -- 172.21.15.62:0/2167426741 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084007740 con 0x7f208c15f760 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f208affd640 1 -- 172.21.15.62:0/2167426741 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20840078e0 con 0x7f208c15f760 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f208affd640 1 -- 172.21.15.62:0/2167426741 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084007cb0 con 0x7f208c15f760 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 -- 172.21.15.62:0/2167426741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c15f760 msgr2=0x7f208c0b9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 --2- 172.21.15.62:0/2167426741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c15f760 0x7f208c0b9190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f208c0d4520 tx=0x7f208402a7a0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 -- 172.21.15.62:0/2167426741 shutdown_connections 2024-04-26T22:28:40.300 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 --2- 172.21.15.62:0/2167426741 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c15f760 0x7f208c0b9190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 -- 172.21.15.62:0/2167426741 >> 172.21.15.62:0/2167426741 conn(0x7f208c09e560 msgr2=0x7f208c0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 -- 172.21.15.62:0/2167426741 shutdown_connections 2024-04-26T22:28:40.301 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.299+0000 7f2092485640 1 -- 172.21.15.62:0/2167426741 wait complete. 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.300+0000 7f2092485640 1 Processor -- start 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 -- start start 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 0x7f208c15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 0x7f208c160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f208c0d1280 con 0x7f208c0d22e0 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f208c0d1100 con 0x7f208c15f760 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.301+0000 7f2092485640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f208c0d5b50 con 0x7f208c15f380 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60708/0 (socket says 172.21.15.62:60708) 2024-04-26T22:28:40.303 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 -- 172.21.15.62:0/3603904015 learned_addr learned my addr 172.21.15.62:0/3603904015 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2090a22640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 0x7f208c160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208b7fe640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 0x7f208c15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 -- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 msgr2=0x7f208c15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 0x7f208c15fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 -- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 msgr2=0x7f208c160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 0x7f208c160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 -- 172.21.15.62:0/3603904015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c0cd450 con 0x7f208c0d22e0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2090a22640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 0x7f208c160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208b7fe640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 0x7f208c15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f208bfff640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c450 tx=0x7f207c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0052d0 con 0x7f208c0d22e0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207c005470 con 0x7f208c0d22e0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c012870 con 0x7f208c0d22e0 2024-04-26T22:28:40.304 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c145190 con 0x7f208c0d22e0 2024-04-26T22:28:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.302+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c162a80 con 0x7f208c0d22e0 2024-04-26T22:28:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.303+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208c06a360 con 0x7f208c0d22e0 2024-04-26T22:28:40.305 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.304+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f207c012a10 con 0x7f208c0d22e0 2024-04-26T22:28:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.304+0000 7f2088ff9640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 0x7f207806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.304+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f207c08b720 con 0x7f208c0d22e0 2024-04-26T22:28:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.304+0000 7f208b7fe640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 0x7f207806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.306 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.305+0000 7f208b7fe640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 0x7f207806ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f208c0cbb40 tx=0x7f2080007c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.308 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.309+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f207c059bf0 con 0x7f208c0d22e0 2024-04-26T22:28:40.489 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.489+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f208c0b7ec0 con 0x7f207806c980 2024-04-26T22:28:40.495 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.495+0000 7f2088ff9640 1 -- 172.21.15.62:0/3603904015 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7f208c0b7ec0 con 0x7f207806c980 2024-04-26T22:28:40.497 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:40.499 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 msgr2=0x7f207806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 0x7f207806ee40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f208c0cbb40 tx=0x7f2080007c50 comp rx=0 tx=0).stop 2024-04-26T22:28:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 msgr2=0x7f208c161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f207c00c450 tx=0x7f207c00c910 comp rx=0 tx=0).stop 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 shutdown_connections 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f207806c980 0x7f207806ee40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f208c15f760 0x7f208c160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f208c15f380 0x7f208c15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 --2- 172.21.15.62:0/3603904015 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f208c0d22e0 0x7f208c161ce0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 >> 172.21.15.62:0/3603904015 conn(0x7f208c09e560 msgr2=0x7f208c0b8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 shutdown_connections 2024-04-26T22:28:40.501 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.498+0000 7f2092485640 1 -- 172.21.15.62:0/3603904015 wait complete. 2024-04-26T22:28:40.509 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-26T22:28:39.658448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221032,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753688,"statfs":{"total":773094113280,"available":772867776512,"internally_reserved":0,"allocated":4972544,"data_stored":2676652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.524630"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148975+0000","last_change":"2024-04-26T22:28:36.148975+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148975+0000","last_undegraded":"2024-04-26T22:28:36.148975+0000","last_fullsized":"2024-04-26T22:28:36.148975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148634+0000","last_change":"2024-04-26T22:28:36.148634+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148634+0000","last_undegraded":"2024-04-26T22:28:36.148634+0000","last_fullsized":"2024-04-26T22:28:36.148634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157768+0000","last_change":"2024-04-26T22:28:36.157768+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157768+0000","last_undegraded":"2024-04-26T22:28:36.157768+0000","last_fullsized":"2024-04-26T22:28:36.157768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157943+0000","last_change":"2024-04-26T22:28:36.157943+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157943+0000","last_undegraded":"2024-04-26T22:28:36.157943+0000","last_fullsized":"2024-04-26T22:28:36.157943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148758+0000","last_change":"2024-04-26T22:28:36.148758+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148758+0000","last_undegraded":"2024-04-26T22:28:36.148758+0000","last_fullsized":"2024-04-26T22:28:36.148758+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151939+0000","last_change":"2024-04-26T22:28:36.151939+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151939+0000","last_undegraded":"2024-04-26T22:28:36.151939+0000","last_fullsized":"2024-04-26T22:28:36.151939+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150312+0000","last_change":"2024-04-26T22:28:36.150312+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150312+0000","last_undegraded":"2024-04-26T22:28:36.150312+0000","last_fullsized":"2024-04-26T22:28:36.150312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157416+0000","last_change":"2024-04-26T22:28:36.157416+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157416+0000","last_undegraded":"2024-04-26T22:28:36.157416+0000","last_fullsized":"2024-04-26T22:28:36.157416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157243+0000","last_change":"2024-04-26T22:28:36.157243+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157243+0000","last_undegraded":"2024-04-26T22:28:36.157243+0000","last_fullsized":"2024-04-26T22:28:36.157243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156453+0000","last_change":"2024-04-26T22:28:36.156453+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156453+0000","last_undegraded":"2024-04-26T22:28:36.156453+0000","last_fullsized":"2024-04-26T22:28:36.156453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154387+0000","last_change":"2024-04-26T22:28:36.154387+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154387+0000","last_undegraded":"2024-04-26T22:28:36.154387+0000","last_fullsized":"2024-04-26T22:28:36.154387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150951+0000","last_change":"2024-04-26T22:28:36.150951+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150951+0000","last_undegraded":"2024-04-26T22:28:36.150951+0000","last_fullsized":"2024-04-26T22:28:36.150951+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147261+0000","last_change":"2024-04-26T22:28:36.147261+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147261+0000","last_undegraded":"2024-04-26T22:28:36.147261+0000","last_fullsized":"2024-04-26T22:28:36.147261+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154697+0000","last_change":"2024-04-26T22:28:36.154697+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154697+0000","last_undegraded":"2024-04-26T22:28:36.154697+0000","last_fullsized":"2024-04-26T22:28:36.154697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152160+0000","last_change":"2024-04-26T22:28:36.152160+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152160+0000","last_undegraded":"2024-04-26T22:28:36.152160+0000","last_fullsized":"2024-04-26T22:28:36.152160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151319+0000","last_change":"2024-04-26T22:28:36.151319+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151319+0000","last_undegraded":"2024-04-26T22:28:36.151319+0000","last_fullsized":"2024-04-26T22:28:36.151319+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157224+0000","last_change":"2024-04-26T22:28:36.157224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157224+0000","last_undegraded":"2024-04-26T22:28:36.157224+0000","last_fullsized":"2024-04-26T22:28:36.157224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.159081+0000","last_change":"2024-04-26T22:28:36.159081+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.159081+0000","last_undegraded":"2024-04-26T22:28:36.159081+0000","last_fullsized":"2024-04-26T22:28:36.159081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152582+0000","last_change":"2024-04-26T22:28:36.152582+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152582+0000","last_undegraded":"2024-04-26T22:28:36.152582+0000","last_fullsized":"2024-04-26T22:28:36.152582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148009+0000","last_change":"2024-04-26T22:28:36.148009+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148009+0000","last_undegraded":"2024-04-26T22:28:36.148009+0000","last_fullsized":"2024-04-26T22:28:36.148009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150983+0000","last_change":"2024-04-26T22:28:36.150983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150983+0000","last_undegraded":"2024-04-26T22:28:36.150983+0000","last_fullsized":"2024-04-26T22:28:36.150983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154224+0000","last_change":"2024-04-26T22:28:36.154224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154224+0000","last_undegraded":"2024-04-26T22:28:36.154224+0000","last_fullsized":"2024-04-26T22:28:36.154224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152240+0000","last_change":"2024-04-26T22:28:36.152240+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152240+0000","last_undegraded":"2024-04-26T22:28:36.152240+0000","last_fullsized":"2024-04-26T22:28:36.152240+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150753+0000","last_change":"2024-04-26T22:28:36.150753+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150753+0000","last_undegraded":"2024-04-26T22:28:36.150753+0000","last_fullsized":"2024-04-26T22:28:36.150753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154621+0000","last_change":"2024-04-26T22:28:36.154621+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154621+0000","last_undegraded":"2024-04-26T22:28:36.154621+0000","last_fullsized":"2024-04-26T22:28:36.154621+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157292+0000","last_change":"2024-04-26T22:28:36.157292+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157292+0000","last_undegraded":"2024-04-26T22:28:36.157292+0000","last_fullsized":"2024-04-26T22:28:36.157292+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150520+0000","last_change":"2024-04-26T22:28:36.150520+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150520+0000","last_undegraded":"2024-04-26T22:28:36.150520+0000","last_fullsized":"2024-04-26T22:28:36.150520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151361+0000","last_change":"2024-04-26T22:28:36.151361+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151361+0000","last_undegraded":"2024-04-26T22:28:36.151361+0000","last_fullsized":"2024-04-26T22:28:36.151361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158720+0000","last_change":"2024-04-26T22:28:36.158720+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158720+0000","last_undegraded":"2024-04-26T22:28:36.158720+0000","last_fullsized":"2024-04-26T22:28:36.158720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150207+0000","last_change":"2024-04-26T22:28:36.150207+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150207+0000","last_undegraded":"2024-04-26T22:28:36.150207+0000","last_fullsized":"2024-04-26T22:28:36.150207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147125+0000","last_change":"2024-04-26T22:28:36.147125+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147125+0000","last_undegraded":"2024-04-26T22:28:36.147125+0000","last_fullsized":"2024-04-26T22:28:36.147125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159896+0000","last_change":"2024-04-26T22:28:36.157128+0000","last_active":"2024-04-26T22:28:36.159896+0000","last_peered":"2024-04-26T22:28:36.159896+0000","last_clean":"2024-04-26T22:28:36.159896+0000","last_became_active":"2024-04-26T22:28:36.156854+0000","last_became_peered":"2024-04-26T22:28:36.156854+0000","last_unstale":"2024-04-26T22:28:36.159896+0000","last_undegraded":"2024-04-26T22:28:36.159896+0000","last_fullsized":"2024-04-26T22:28:36.159896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154011+0000","last_change":"2024-04-26T22:28:36.154011+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154011+0000","last_undegraded":"2024-04-26T22:28:36.154011+0000","last_fullsized":"2024-04-26T22:28:36.154011+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170598+0000","last_change":"2024-04-26T22:28:36.165408+0000","last_active":"2024-04-26T22:28:36.170598+0000","last_peered":"2024-04-26T22:28:36.170598+0000","last_clean":"2024-04-26T22:28:36.170598+0000","last_became_active":"2024-04-26T22:28:36.165110+0000","last_became_peered":"2024-04-26T22:28:36.165110+0000","last_unstale":"2024-04-26T22:28:36.170598+0000","last_undegraded":"2024-04-26T22:28:36.170598+0000","last_fullsized":"2024-04-26T22:28:36.170598+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148308+0000","last_change":"2024-04-26T22:28:36.148308+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148308+0000","last_undegraded":"2024-04-26T22:28:36.148308+0000","last_fullsized":"2024-04-26T22:28:36.148308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.156144+0000","last_change":"2024-04-26T22:28:36.154806+0000","last_active":"2024-04-26T22:28:36.156144+0000","last_peered":"2024-04-26T22:28:36.156144+0000","last_clean":"2024-04-26T22:28:36.156144+0000","last_became_active":"2024-04-26T22:28:36.154497+0000","last_became_peered":"2024-04-26T22:28:36.154497+0000","last_unstale":"2024-04-26T22:28:36.156144+0000","last_undegraded":"2024-04-26T22:28:36.156144+0000","last_fullsized":"2024-04-26T22:28:36.156144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148250+0000","last_change":"2024-04-26T22:28:36.148250+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148250+0000","last_undegraded":"2024-04-26T22:28:36.148250+0000","last_fullsized":"2024-04-26T22:28:36.148250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170452+0000","last_change":"2024-04-26T22:28:36.165313+0000","last_active":"2024-04-26T22:28:36.170452+0000","last_peered":"2024-04-26T22:28:36.170452+0000","last_clean":"2024-04-26T22:28:36.170452+0000","last_became_active":"2024-04-26T22:28:36.165008+0000","last_became_peered":"2024-04-26T22:28:36.165008+0000","last_unstale":"2024-04-26T22:28:36.170452+0000","last_undegraded":"2024-04-26T22:28:36.170452+0000","last_fullsized":"2024-04-26T22:28:36.170452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:56.424170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157367+0000","last_change":"2024-04-26T22:28:36.157367+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157367+0000","last_undegraded":"2024-04-26T22:28:36.157367+0000","last_fullsized":"2024-04-26T22:28:36.157367+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159950+0000","last_change":"2024-04-26T22:28:36.157297+0000","last_active":"2024-04-26T22:28:36.159950+0000","last_peered":"2024-04-26T22:28:36.159950+0000","last_clean":"2024-04-26T22:28:36.159950+0000","last_became_active":"2024-04-26T22:28:36.157002+0000","last_became_peered":"2024-04-26T22:28:36.157002+0000","last_unstale":"2024-04-26T22:28:36.159950+0000","last_undegraded":"2024-04-26T22:28:36.159950+0000","last_fullsized":"2024-04-26T22:28:36.159950+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:36.488188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155983+0000","last_change":"2024-04-26T22:28:36.155983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155983+0000","last_undegraded":"2024-04-26T22:28:36.155983+0000","last_fullsized":"2024-04-26T22:28:36.155983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.172286+0000","last_change":"2024-04-26T22:28:36.167783+0000","last_active":"2024-04-26T22:28:36.172286+0000","last_peered":"2024-04-26T22:28:36.172286+0000","last_clean":"2024-04-26T22:28:36.172286+0000","last_became_active":"2024-04-26T22:28:36.167480+0000","last_became_peered":"2024-04-26T22:28:36.167480+0000","last_unstale":"2024-04-26T22:28:36.172286+0000","last_undegraded":"2024-04-26T22:28:36.172286+0000","last_fullsized":"2024-04-26T22:28:36.172286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152072+0000","last_change":"2024-04-26T22:28:36.152072+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152072+0000","last_undegraded":"2024-04-26T22:28:36.152072+0000","last_fullsized":"2024-04-26T22:28:36.152072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167154+0000","last_change":"2024-04-26T22:28:36.161210+0000","last_active":"2024-04-26T22:28:36.167154+0000","last_peered":"2024-04-26T22:28:36.167154+0000","last_clean":"2024-04-26T22:28:36.167154+0000","last_became_active":"2024-04-26T22:28:36.160683+0000","last_became_peered":"2024-04-26T22:28:36.160683+0000","last_unstale":"2024-04-26T22:28:36.167154+0000","last_undegraded":"2024-04-26T22:28:36.167154+0000","last_fullsized":"2024-04-26T22:28:36.167154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153574+0000","last_change":"2024-04-26T22:28:36.153574+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153574+0000","last_undegraded":"2024-04-26T22:28:36.153574+0000","last_fullsized":"2024-04-26T22:28:36.153574+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164527+0000","last_change":"2024-04-26T22:28:36.162766+0000","last_active":"2024-04-26T22:28:36.164527+0000","last_peered":"2024-04-26T22:28:36.164527+0000","last_clean":"2024-04-26T22:28:36.164527+0000","last_became_active":"2024-04-26T22:28:36.162468+0000","last_became_peered":"2024-04-26T22:28:36.162468+0000","last_unstale":"2024-04-26T22:28:36.164527+0000","last_undegraded":"2024-04-26T22:28:36.164527+0000","last_fullsized":"2024-04-26T22:28:36.164527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158533+0000","last_change":"2024-04-26T22:28:36.158533+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158533+0000","last_undegraded":"2024-04-26T22:28:36.158533+0000","last_fullsized":"2024-04-26T22:28:36.158533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164922+0000","last_change":"2024-04-26T22:28:36.162336+0000","last_active":"2024-04-26T22:28:36.164922+0000","last_peered":"2024-04-26T22:28:36.164922+0000","last_clean":"2024-04-26T22:28:36.164922+0000","last_became_active":"2024-04-26T22:28:36.162067+0000","last_became_peered":"2024-04-26T22:28:36.162067+0000","last_unstale":"2024-04-26T22:28:36.164922+0000","last_undegraded":"2024-04-26T22:28:36.164922+0000","last_fullsized":"2024-04-26T22:28:36.164922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148121+0000","last_change":"2024-04-26T22:28:36.148121+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148121+0000","last_undegraded":"2024-04-26T22:28:36.148121+0000","last_fullsized":"2024-04-26T22:28:36.148121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164607+0000","last_change":"2024-04-26T22:28:36.161738+0000","last_active":"2024-04-26T22:28:36.164607+0000","last_peered":"2024-04-26T22:28:36.164607+0000","last_clean":"2024-04-26T22:28:36.164607+0000","last_became_active":"2024-04-26T22:28:36.161489+0000","last_became_peered":"2024-04-26T22:28:36.161489+0000","last_unstale":"2024-04-26T22:28:36.164607+0000","last_undegraded":"2024-04-26T22:28:36.164607+0000","last_fullsized":"2024-04-26T22:28:36.164607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150196+0000","last_change":"2024-04-26T22:28:36.150196+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150196+0000","last_undegraded":"2024-04-26T22:28:36.150196+0000","last_fullsized":"2024-04-26T22:28:36.150196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164669+0000","last_change":"2024-04-26T22:28:36.159715+0000","last_active":"2024-04-26T22:28:36.164669+0000","last_peered":"2024-04-26T22:28:36.164669+0000","last_clean":"2024-04-26T22:28:36.164669+0000","last_became_active":"2024-04-26T22:28:36.159437+0000","last_became_peered":"2024-04-26T22:28:36.159437+0000","last_unstale":"2024-04-26T22:28:36.164669+0000","last_undegraded":"2024-04-26T22:28:36.164669+0000","last_fullsized":"2024-04-26T22:28:36.164669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149529+0000","last_change":"2024-04-26T22:28:36.149529+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149529+0000","last_undegraded":"2024-04-26T22:28:36.149529+0000","last_fullsized":"2024-04-26T22:28:36.149529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165270+0000","last_change":"2024-04-26T22:28:36.163369+0000","last_active":"2024-04-26T22:28:36.165270+0000","last_peered":"2024-04-26T22:28:36.165270+0000","last_clean":"2024-04-26T22:28:36.165270+0000","last_became_active":"2024-04-26T22:28:36.163060+0000","last_became_peered":"2024-04-26T22:28:36.163060+0000","last_unstale":"2024-04-26T22:28:36.165270+0000","last_undegraded":"2024-04-26T22:28:36.165270+0000","last_fullsized":"2024-04-26T22:28:36.165270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:58.196234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151707+0000","last_change":"2024-04-26T22:28:36.151707+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151707+0000","last_undegraded":"2024-04-26T22:28:36.151707+0000","last_fullsized":"2024-04-26T22:28:36.151707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.166646+0000","last_change":"2024-04-26T22:28:36.165081+0000","last_active":"2024-04-26T22:28:36.166646+0000","last_peered":"2024-04-26T22:28:36.166646+0000","last_clean":"2024-04-26T22:28:36.166646+0000","last_became_active":"2024-04-26T22:28:36.164792+0000","last_became_peered":"2024-04-26T22:28:36.164792+0000","last_unstale":"2024-04-26T22:28:36.166646+0000","last_undegraded":"2024-04-26T22:28:36.166646+0000","last_fullsized":"2024-04-26T22:28:36.166646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:54.680123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155483+0000","last_change":"2024-04-26T22:28:36.155483+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155483+0000","last_undegraded":"2024-04-26T22:28:36.155483+0000","last_fullsized":"2024-04-26T22:28:36.155483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162204+0000","last_change":"2024-04-26T22:28:36.158147+0000","last_active":"2024-04-26T22:28:36.162204+0000","last_peered":"2024-04-26T22:28:36.162204+0000","last_clean":"2024-04-26T22:28:36.162204+0000","last_became_active":"2024-04-26T22:28:36.157890+0000","last_became_peered":"2024-04-26T22:28:36.157890+0000","last_unstale":"2024-04-26T22:28:36.162204+0000","last_undegraded":"2024-04-26T22:28:36.162204+0000","last_fullsized":"2024-04-26T22:28:36.162204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151703+0000","last_change":"2024-04-26T22:28:36.151703+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151703+0000","last_undegraded":"2024-04-26T22:28:36.151703+0000","last_fullsized":"2024-04-26T22:28:36.151703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165776+0000","last_change":"2024-04-26T22:28:36.164416+0000","last_active":"2024-04-26T22:28:36.165776+0000","last_peered":"2024-04-26T22:28:36.165776+0000","last_clean":"2024-04-26T22:28:36.165776+0000","last_became_active":"2024-04-26T22:28:36.164074+0000","last_became_peered":"2024-04-26T22:28:36.164074+0000","last_unstale":"2024-04-26T22:28:36.165776+0000","last_undegraded":"2024-04-26T22:28:36.165776+0000","last_fullsized":"2024-04-26T22:28:36.165776+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:44.778533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148440+0000","last_change":"2024-04-26T22:28:36.148440+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148440+0000","last_undegraded":"2024-04-26T22:28:36.148440+0000","last_fullsized":"2024-04-26T22:28:36.148440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.168241+0000","last_change":"2024-04-26T22:28:36.167663+0000","last_active":"2024-04-26T22:28:36.168241+0000","last_peered":"2024-04-26T22:28:36.168241+0000","last_clean":"2024-04-26T22:28:36.168241+0000","last_became_active":"2024-04-26T22:28:36.167430+0000","last_became_peered":"2024-04-26T22:28:36.167430+0000","last_unstale":"2024-04-26T22:28:36.168241+0000","last_undegraded":"2024-04-26T22:28:36.168241+0000","last_fullsized":"2024-04-26T22:28:36.168241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146555+0000","last_change":"2024-04-26T22:28:36.146555+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146555+0000","last_undegraded":"2024-04-26T22:28:36.146555+0000","last_fullsized":"2024-04-26T22:28:36.146555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155478+0000","last_change":"2024-04-26T22:28:36.152997+0000","last_active":"2024-04-26T22:28:36.155478+0000","last_peered":"2024-04-26T22:28:36.155478+0000","last_clean":"2024-04-26T22:28:36.155478+0000","last_became_active":"2024-04-26T22:28:36.152694+0000","last_became_peered":"2024-04-26T22:28:36.152694+0000","last_unstale":"2024-04-26T22:28:36.155478+0000","last_undegraded":"2024-04-26T22:28:36.155478+0000","last_fullsized":"2024-04-26T22:28:36.155478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:42.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153502+0000","last_change":"2024-04-26T22:28:36.153502+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153502+0000","last_undegraded":"2024-04-26T22:28:36.153502+0000","last_fullsized":"2024-04-26T22:28:36.153502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164982+0000","last_change":"2024-04-26T22:28:36.162940+0000","last_active":"2024-04-26T22:28:36.164982+0000","last_peered":"2024-04-26T22:28:36.164982+0000","last_clean":"2024-04-26T22:28:36.164982+0000","last_became_active":"2024-04-26T22:28:36.162676+0000","last_became_peered":"2024-04-26T22:28:36.162676+0000","last_unstale":"2024-04-26T22:28:36.164982+0000","last_undegraded":"2024-04-26T22:28:36.164982+0000","last_fullsized":"2024-04-26T22:28:36.164982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153988+0000","last_change":"2024-04-26T22:28:36.153988+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153988+0000","last_undegraded":"2024-04-26T22:28:36.153988+0000","last_fullsized":"2024-04-26T22:28:36.153988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167075+0000","last_change":"2024-04-26T22:28:36.157167+0000","last_active":"2024-04-26T22:28:36.167075+0000","last_peered":"2024-04-26T22:28:36.167075+0000","last_clean":"2024-04-26T22:28:36.167075+0000","last_became_active":"2024-04-26T22:28:36.156867+0000","last_became_peered":"2024-04-26T22:28:36.156867+0000","last_unstale":"2024-04-26T22:28:36.167075+0000","last_undegraded":"2024-04-26T22:28:36.167075+0000","last_fullsized":"2024-04-26T22:28:36.167075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154215+0000","last_change":"2024-04-26T22:28:36.154215+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154215+0000","last_undegraded":"2024-04-26T22:28:36.154215+0000","last_fullsized":"2024-04-26T22:28:36.154215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.163612+0000","last_change":"2024-04-26T22:28:36.160489+0000","last_active":"2024-04-26T22:28:36.163612+0000","last_peered":"2024-04-26T22:28:36.163612+0000","last_clean":"2024-04-26T22:28:36.163612+0000","last_became_active":"2024-04-26T22:28:36.160227+0000","last_became_peered":"2024-04-26T22:28:36.160227+0000","last_unstale":"2024-04-26T22:28:36.163612+0000","last_undegraded":"2024-04-26T22:28:36.163612+0000","last_fullsized":"2024-04-26T22:28:36.163612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:23.581702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149391+0000","last_change":"2024-04-26T22:28:36.149391+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149391+0000","last_undegraded":"2024-04-26T22:28:36.149391+0000","last_fullsized":"2024-04-26T22:28:36.149391+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159929+0000","last_change":"2024-04-26T22:28:36.156431+0000","last_active":"2024-04-26T22:28:36.159929+0000","last_peered":"2024-04-26T22:28:36.159929+0000","last_clean":"2024-04-26T22:28:36.159929+0000","last_became_active":"2024-04-26T22:28:36.156129+0000","last_became_peered":"2024-04-26T22:28:36.156129+0000","last_unstale":"2024-04-26T22:28:36.159929+0000","last_undegraded":"2024-04-26T22:28:36.159929+0000","last_fullsized":"2024-04-26T22:28:36.159929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:07.246183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146638+0000","last_change":"2024-04-26T22:28:36.146638+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146638+0000","last_undegraded":"2024-04-26T22:28:36.146638+0000","last_fullsized":"2024-04-26T22:28:36.146638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167053+0000","last_change":"2024-04-26T22:28:36.156716+0000","last_active":"2024-04-26T22:28:36.167053+0000","last_peered":"2024-04-26T22:28:36.167053+0000","last_clean":"2024-04-26T22:28:36.167053+0000","last_became_active":"2024-04-26T22:28:36.156424+0000","last_became_peered":"2024-04-26T22:28:36.156424+0000","last_unstale":"2024-04-26T22:28:36.167053+0000","last_undegraded":"2024-04-26T22:28:36.167053+0000","last_fullsized":"2024-04-26T22:28:36.167053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146530+0000","last_change":"2024-04-26T22:28:36.146530+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146530+0000","last_undegraded":"2024-04-26T22:28:36.146530+0000","last_fullsized":"2024-04-26T22:28:36.146530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.160043+0000","last_change":"2024-04-26T22:28:36.157532+0000","last_active":"2024-04-26T22:28:36.160043+0000","last_peered":"2024-04-26T22:28:36.160043+0000","last_clean":"2024-04-26T22:28:36.160043+0000","last_became_active":"2024-04-26T22:28:36.157238+0000","last_became_peered":"2024-04-26T22:28:36.157238+0000","last_unstale":"2024-04-26T22:28:36.160043+0000","last_undegraded":"2024-04-26T22:28:36.160043+0000","last_fullsized":"2024-04-26T22:28:36.160043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:16.612120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151105+0000","last_change":"2024-04-26T22:28:36.151105+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151105+0000","last_undegraded":"2024-04-26T22:28:36.151105+0000","last_fullsized":"2024-04-26T22:28:36.151105+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162295+0000","last_change":"2024-04-26T22:28:36.157860+0000","last_active":"2024-04-26T22:28:36.162295+0000","last_peered":"2024-04-26T22:28:36.162295+0000","last_clean":"2024-04-26T22:28:36.162295+0000","last_became_active":"2024-04-26T22:28:36.157588+0000","last_became_peered":"2024-04-26T22:28:36.157588+0000","last_unstale":"2024-04-26T22:28:36.162295+0000","last_undegraded":"2024-04-26T22:28:36.162295+0000","last_fullsized":"2024-04-26T22:28:36.162295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153545+0000","last_change":"2024-04-26T22:28:36.153545+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153545+0000","last_undegraded":"2024-04-26T22:28:36.153545+0000","last_fullsized":"2024-04-26T22:28:36.153545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167355+0000","last_change":"2024-04-26T22:28:36.165811+0000","last_active":"2024-04-26T22:28:36.167355+0000","last_peered":"2024-04-26T22:28:36.167355+0000","last_clean":"2024-04-26T22:28:36.167355+0000","last_became_active":"2024-04-26T22:28:36.165461+0000","last_became_peered":"2024-04-26T22:28:36.165461+0000","last_unstale":"2024-04-26T22:28:36.167355+0000","last_undegraded":"2024-04-26T22:28:36.167355+0000","last_fullsized":"2024-04-26T22:28:36.167355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:18.602353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153458+0000","last_change":"2024-04-26T22:28:36.153458+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153458+0000","last_undegraded":"2024-04-26T22:28:36.153458+0000","last_fullsized":"2024-04-26T22:28:36.153458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164692+0000","last_change":"2024-04-26T22:28:36.159775+0000","last_active":"2024-04-26T22:28:36.164692+0000","last_peered":"2024-04-26T22:28:36.164692+0000","last_clean":"2024-04-26T22:28:36.164692+0000","last_became_active":"2024-04-26T22:28:36.159490+0000","last_became_peered":"2024-04-26T22:28:36.159490+0000","last_unstale":"2024-04-26T22:28:36.164692+0000","last_undegraded":"2024-04-26T22:28:36.164692+0000","last_fullsized":"2024-04-26T22:28:36.164692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149079+0000","last_change":"2024-04-26T22:28:36.149079+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149079+0000","last_undegraded":"2024-04-26T22:28:36.149079+0000","last_fullsized":"2024-04-26T22:28:36.149079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155941+0000","last_change":"2024-04-26T22:28:36.154743+0000","last_active":"2024-04-26T22:28:36.155941+0000","last_peered":"2024-04-26T22:28:36.155941+0000","last_clean":"2024-04-26T22:28:36.155941+0000","last_became_active":"2024-04-26T22:28:36.154433+0000","last_became_peered":"2024-04-26T22:28:36.154433+0000","last_unstale":"2024-04-26T22:28:36.155941+0000","last_undegraded":"2024-04-26T22:28:36.155941+0000","last_fullsized":"2024-04-26T22:28:36.155941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146336+0000","last_change":"2024-04-26T22:28:36.146336+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146336+0000","last_undegraded":"2024-04-26T22:28:36.146336+0000","last_fullsized":"2024-04-26T22:28:36.146336+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164732+0000","last_change":"2024-04-26T22:28:36.160465+0000","last_active":"2024-04-26T22:28:36.164732+0000","last_peered":"2024-04-26T22:28:36.164732+0000","last_clean":"2024-04-26T22:28:36.164732+0000","last_became_active":"2024-04-26T22:28:36.160206+0000","last_became_peered":"2024-04-26T22:28:36.160206+0000","last_unstale":"2024-04-26T22:28:36.164732+0000","last_undegraded":"2024-04-26T22:28:36.164732+0000","last_fullsized":"2024-04-26T22:28:36.164732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:47.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146383+0000","last_change":"2024-04-26T22:28:36.146383+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146383+0000","last_undegraded":"2024-04-26T22:28:36.146383+0000","last_fullsized":"2024-04-26T22:28:36.146383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164966+0000","last_change":"2024-04-26T22:28:36.163006+0000","last_active":"2024-04-26T22:28:36.164966+0000","last_peered":"2024-04-26T22:28:36.164966+0000","last_clean":"2024-04-26T22:28:36.164966+0000","last_became_active":"2024-04-26T22:28:36.162735+0000","last_became_peered":"2024-04-26T22:28:36.162735+0000","last_unstale":"2024-04-26T22:28:36.164966+0000","last_undegraded":"2024-04-26T22:28:36.164966+0000","last_fullsized":"2024-04-26T22:28:36.164966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147713+0000","last_change":"2024-04-26T22:28:36.147713+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147713+0000","last_undegraded":"2024-04-26T22:28:36.147713+0000","last_fullsized":"2024-04-26T22:28:36.147713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.161202+0000","last_change":"2024-04-26T22:28:36.159382+0000","last_active":"2024-04-26T22:28:36.161202+0000","last_peered":"2024-04-26T22:28:36.161202+0000","last_clean":"2024-04-26T22:28:36.161202+0000","last_became_active":"2024-04-26T22:28:36.159016+0000","last_became_peered":"2024-04-26T22:28:36.159016+0000","last_unstale":"2024-04-26T22:28:36.161202+0000","last_undegraded":"2024-04-26T22:28:36.161202+0000","last_fullsized":"2024-04-26T22:28:36.161202+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156626+0000","last_change":"2024-04-26T22:28:36.156626+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156626+0000","last_undegraded":"2024-04-26T22:28:36.156626+0000","last_fullsized":"2024-04-26T22:28:36.156626+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164507+0000","last_change":"2024-04-26T22:28:36.162470+0000","last_active":"2024-04-26T22:28:36.164507+0000","last_peered":"2024-04-26T22:28:36.164507+0000","last_clean":"2024-04-26T22:28:36.164507+0000","last_became_active":"2024-04-26T22:28:36.162164+0000","last_became_peered":"2024-04-26T22:28:36.162164+0000","last_unstale":"2024-04-26T22:28:36.164507+0000","last_undegraded":"2024-04-26T22:28:36.164507+0000","last_fullsized":"2024-04-26T22:28:36.164507+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155799+0000","last_change":"2024-04-26T22:28:36.155799+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155799+0000","last_undegraded":"2024-04-26T22:28:36.155799+0000","last_fullsized":"2024-04-26T22:28:36.155799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167245+0000","last_change":"2024-04-26T22:28:36.162279+0000","last_active":"2024-04-26T22:28:36.167245+0000","last_peered":"2024-04-26T22:28:36.167245+0000","last_clean":"2024-04-26T22:28:36.167245+0000","last_became_active":"2024-04-26T22:28:36.161730+0000","last_became_peered":"2024-04-26T22:28:36.161730+0000","last_unstale":"2024-04-26T22:28:36.167245+0000","last_undegraded":"2024-04-26T22:28:36.167245+0000","last_fullsized":"2024-04-26T22:28:36.167245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.152298+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:36.152298+0000","last_peered":"2024-04-26T22:28:36.152298+0000","last_clean":"2024-04-26T22:28:36.152298+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:36.152298+0000","last_undegraded":"2024-04-26T22:28:36.152298+0000","last_fullsized":"2024-04-26T22:28:36.152298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":774224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:40.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:40.649 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.649+0000 7fc0e6c93640 1 Processor -- start 2024-04-26T22:28:40.650 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.649+0000 7fc0e6c93640 1 -- start start 2024-04-26T22:28:40.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0e6c93640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0e00d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0e6c93640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc0e00b9a90 0x7fc0e00d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0e6c93640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0e00d3de0 con 0x7fc0e015f760 2024-04-26T22:28:40.652 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0e6c93640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc0e00d43a0 con 0x7fc0e00b9a90 2024-04-26T22:28:40.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0e6c93640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e00d40c0 con 0x7fc0e015f380 2024-04-26T22:28:40.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0dffff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fc0e015f760 0x7fc0e00cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56626/0 (socket says 172.21.15.62:56626) 2024-04-26T22:28:40.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.650+0000 7fc0dffff640 1 -- 172.21.15.62:0/2234252862 learned_addr learned my addr 172.21.15.62:0/2234252862 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:40.653 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e4a2f640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0e00d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2234252862 >> v2:172.21.15.174:3300/0 conn(0x7fc0e00b9a90 0x7fc0e00d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384531483 0 0) 0x7fc0e00d3de0 con 0x7fc0e015f760 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0d0003520 con 0x7fc0e015f760 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1238975089 0 0) 0x7fc0d0003520 con 0x7fc0e015f760 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 msgr2=0x7fc0e00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0e00d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 >> v2:172.21.15.174:3300/0 conn(0x7fc0e00b9a90 msgr2=0x7fc0e00d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 --2- 172.21.15.62:0/2234252862 >> v2:172.21.15.174:3300/0 conn(0x7fc0e00b9a90 0x7fc0e00d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e00b8280 con 0x7fc0e015f760 2024-04-26T22:28:40.654 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fc0cc001670 con 0x7fc0e015f760 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 >> v1:172.21.15.62:6789/0 conn(0x7fc0e015f760 legacy=0x7fc0e00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0d00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d0003f80 0x7fc0d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00d43a0 con 0x7fc0d0004d40 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00d40c0 con 0x7fc0d0003f80 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d000ba40 con 0x7fc0e015f380 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0dffff640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d0003f80 0x7fc0d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e4a2f640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0d00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 -- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 msgr2=0x7fc0d00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0e015f380 0x7fc0d00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 -- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d0003f80 msgr2=0x7fc0d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:40.655 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d0003f80 0x7fc0d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.651+0000 7fc0e5230640 1 -- 172.21.15.62:0/2234252862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d0003520 con 0x7fc0d0004d40 2024-04-26T22:28:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.652+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009fc0 tx=0x7fc0d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fb949b6fc6e8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.652+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8009c10 con 0x7fc0d0004d40 2024-04-26T22:28:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.652+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d8036070 con 0x7fc0d0004d40 2024-04-26T22:28:40.656 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.652+0000 7fc0df7fe640 1 -- 172.21.15.62:0/2234252862 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d802d070 con 0x7fc0d0004d40 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2234252862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d0004d40 msgr2=0x7fc0d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2234252862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0d0004d40 0x7fc0d00051c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009fc0 tx=0x7fc0d0005830 comp rx=0 tx=0).stop 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e5230640 1 -- 172.21.15.62:0/2234252862 reap_dead start 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2234252862 shutdown_connections 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2234252862 >> 172.21.15.62:0/2234252862 conn(0x7fc0e009e560 msgr2=0x7fc0e00d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2234252862 shutdown_connections 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.653+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2234252862 wait complete. 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.654+0000 7fc0e6c93640 1 Processor -- start 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.654+0000 7fc0e6c93640 1 -- start start 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0d0004d40 0x7fc0e00c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0e00cab40 0x7fc0e01f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.659 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00d1280 con 0x7fc0e01f5000 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e00d1100 con 0x7fc0e00cab40 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e6c93640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc0e00d5b50 con 0x7fc0d0004d40 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60714/0 (socket says 172.21.15.62:60714) 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 -- 172.21.15.62:0/2363588983 learned_addr learned my addr 172.21.15.62:0/2363588983 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e4a2f640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0d0004d40 0x7fc0e00c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0dffff640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0e00cab40 0x7fc0e01f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 -- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0d0004d40 msgr2=0x7fc0e00c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0d0004d40 0x7fc0e00c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 -- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0e00cab40 msgr2=0x7fc0e01f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0e00cab40 0x7fc0e01f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.655+0000 7fc0e5230640 1 -- 172.21.15.62:0/2363588983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e00ca1e0 con 0x7fc0e01f5000 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0e5230640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8033040 tx=0x7fc0d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.660 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8009c10 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d8003990 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e00cb100 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8003d80 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.656+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e00cb6c0 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.657+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0e006a360 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.657+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fc0d8005d10 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.658+0000 7fc0dd7fa640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 0x7fc0c406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.658+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fc0d80ad510 con 0x7fc0e01f5000 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.658+0000 7fc0e4a2f640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 0x7fc0c406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:40.661 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.658+0000 7fc0e4a2f640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 0x7fc0c406ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009b00 tx=0x7fc0d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:40.662 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.662+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0d804a060 con 0x7fc0e01f5000 2024-04-26T22:28:40.844 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.844+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0e0076020 con 0x7fc0c406c980 2024-04-26T22:28:40.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.850+0000 7fc0dd7fa640 1 -- 172.21.15.62:0/2363588983 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7fc0e0076020 con 0x7fc0c406c980 2024-04-26T22:28:40.853 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:40.854 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:40.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.853+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 msgr2=0x7fc0c406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.853+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 0x7fc0c406ee40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009b00 tx=0x7fc0d4008040 comp rx=0 tx=0).stop 2024-04-26T22:28:40.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.853+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 msgr2=0x7fc0e01f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:40.855 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.853+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8033040 tx=0x7fc0d8009750 comp rx=0 tx=0).stop 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 shutdown_connections 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fc0c406c980 0x7fc0c406ee40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0e01f5000 0x7fc0e01f5480 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0e00cab40 0x7fc0e01f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 --2- 172.21.15.62:0/2363588983 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0d0004d40 0x7fc0e00c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 >> 172.21.15.62:0/2363588983 conn(0x7fc0e009e560 msgr2=0x7fc0e00b9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 shutdown_connections 2024-04-26T22:28:40.856 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:40.854+0000 7fc0e6c93640 1 -- 172.21.15.62:0/2363588983 wait complete. 2024-04-26T22:28:40.865 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-26T22:28:39.658448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221032,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753688,"statfs":{"total":773094113280,"available":772867776512,"internally_reserved":0,"allocated":4972544,"data_stored":2676652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.524630"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148975+0000","last_change":"2024-04-26T22:28:36.148975+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148975+0000","last_undegraded":"2024-04-26T22:28:36.148975+0000","last_fullsized":"2024-04-26T22:28:36.148975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148634+0000","last_change":"2024-04-26T22:28:36.148634+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148634+0000","last_undegraded":"2024-04-26T22:28:36.148634+0000","last_fullsized":"2024-04-26T22:28:36.148634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157768+0000","last_change":"2024-04-26T22:28:36.157768+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157768+0000","last_undegraded":"2024-04-26T22:28:36.157768+0000","last_fullsized":"2024-04-26T22:28:36.157768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157943+0000","last_change":"2024-04-26T22:28:36.157943+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157943+0000","last_undegraded":"2024-04-26T22:28:36.157943+0000","last_fullsized":"2024-04-26T22:28:36.157943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148758+0000","last_change":"2024-04-26T22:28:36.148758+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148758+0000","last_undegraded":"2024-04-26T22:28:36.148758+0000","last_fullsized":"2024-04-26T22:28:36.148758+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151939+0000","last_change":"2024-04-26T22:28:36.151939+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151939+0000","last_undegraded":"2024-04-26T22:28:36.151939+0000","last_fullsized":"2024-04-26T22:28:36.151939+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150312+0000","last_change":"2024-04-26T22:28:36.150312+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150312+0000","last_undegraded":"2024-04-26T22:28:36.150312+0000","last_fullsized":"2024-04-26T22:28:36.150312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157416+0000","last_change":"2024-04-26T22:28:36.157416+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157416+0000","last_undegraded":"2024-04-26T22:28:36.157416+0000","last_fullsized":"2024-04-26T22:28:36.157416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157243+0000","last_change":"2024-04-26T22:28:36.157243+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157243+0000","last_undegraded":"2024-04-26T22:28:36.157243+0000","last_fullsized":"2024-04-26T22:28:36.157243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156453+0000","last_change":"2024-04-26T22:28:36.156453+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156453+0000","last_undegraded":"2024-04-26T22:28:36.156453+0000","last_fullsized":"2024-04-26T22:28:36.156453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154387+0000","last_change":"2024-04-26T22:28:36.154387+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154387+0000","last_undegraded":"2024-04-26T22:28:36.154387+0000","last_fullsized":"2024-04-26T22:28:36.154387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150951+0000","last_change":"2024-04-26T22:28:36.150951+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150951+0000","last_undegraded":"2024-04-26T22:28:36.150951+0000","last_fullsized":"2024-04-26T22:28:36.150951+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147261+0000","last_change":"2024-04-26T22:28:36.147261+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147261+0000","last_undegraded":"2024-04-26T22:28:36.147261+0000","last_fullsized":"2024-04-26T22:28:36.147261+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154697+0000","last_change":"2024-04-26T22:28:36.154697+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154697+0000","last_undegraded":"2024-04-26T22:28:36.154697+0000","last_fullsized":"2024-04-26T22:28:36.154697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152160+0000","last_change":"2024-04-26T22:28:36.152160+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152160+0000","last_undegraded":"2024-04-26T22:28:36.152160+0000","last_fullsized":"2024-04-26T22:28:36.152160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151319+0000","last_change":"2024-04-26T22:28:36.151319+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151319+0000","last_undegraded":"2024-04-26T22:28:36.151319+0000","last_fullsized":"2024-04-26T22:28:36.151319+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157224+0000","last_change":"2024-04-26T22:28:36.157224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157224+0000","last_undegraded":"2024-04-26T22:28:36.157224+0000","last_fullsized":"2024-04-26T22:28:36.157224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.159081+0000","last_change":"2024-04-26T22:28:36.159081+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.159081+0000","last_undegraded":"2024-04-26T22:28:36.159081+0000","last_fullsized":"2024-04-26T22:28:36.159081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152582+0000","last_change":"2024-04-26T22:28:36.152582+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152582+0000","last_undegraded":"2024-04-26T22:28:36.152582+0000","last_fullsized":"2024-04-26T22:28:36.152582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148009+0000","last_change":"2024-04-26T22:28:36.148009+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148009+0000","last_undegraded":"2024-04-26T22:28:36.148009+0000","last_fullsized":"2024-04-26T22:28:36.148009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150983+0000","last_change":"2024-04-26T22:28:36.150983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150983+0000","last_undegraded":"2024-04-26T22:28:36.150983+0000","last_fullsized":"2024-04-26T22:28:36.150983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154224+0000","last_change":"2024-04-26T22:28:36.154224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154224+0000","last_undegraded":"2024-04-26T22:28:36.154224+0000","last_fullsized":"2024-04-26T22:28:36.154224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152240+0000","last_change":"2024-04-26T22:28:36.152240+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152240+0000","last_undegraded":"2024-04-26T22:28:36.152240+0000","last_fullsized":"2024-04-26T22:28:36.152240+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150753+0000","last_change":"2024-04-26T22:28:36.150753+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150753+0000","last_undegraded":"2024-04-26T22:28:36.150753+0000","last_fullsized":"2024-04-26T22:28:36.150753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154621+0000","last_change":"2024-04-26T22:28:36.154621+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154621+0000","last_undegraded":"2024-04-26T22:28:36.154621+0000","last_fullsized":"2024-04-26T22:28:36.154621+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157292+0000","last_change":"2024-04-26T22:28:36.157292+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157292+0000","last_undegraded":"2024-04-26T22:28:36.157292+0000","last_fullsized":"2024-04-26T22:28:36.157292+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150520+0000","last_change":"2024-04-26T22:28:36.150520+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150520+0000","last_undegraded":"2024-04-26T22:28:36.150520+0000","last_fullsized":"2024-04-26T22:28:36.150520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151361+0000","last_change":"2024-04-26T22:28:36.151361+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151361+0000","last_undegraded":"2024-04-26T22:28:36.151361+0000","last_fullsized":"2024-04-26T22:28:36.151361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158720+0000","last_change":"2024-04-26T22:28:36.158720+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158720+0000","last_undegraded":"2024-04-26T22:28:36.158720+0000","last_fullsized":"2024-04-26T22:28:36.158720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150207+0000","last_change":"2024-04-26T22:28:36.150207+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150207+0000","last_undegraded":"2024-04-26T22:28:36.150207+0000","last_fullsized":"2024-04-26T22:28:36.150207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147125+0000","last_change":"2024-04-26T22:28:36.147125+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147125+0000","last_undegraded":"2024-04-26T22:28:36.147125+0000","last_fullsized":"2024-04-26T22:28:36.147125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159896+0000","last_change":"2024-04-26T22:28:36.157128+0000","last_active":"2024-04-26T22:28:36.159896+0000","last_peered":"2024-04-26T22:28:36.159896+0000","last_clean":"2024-04-26T22:28:36.159896+0000","last_became_active":"2024-04-26T22:28:36.156854+0000","last_became_peered":"2024-04-26T22:28:36.156854+0000","last_unstale":"2024-04-26T22:28:36.159896+0000","last_undegraded":"2024-04-26T22:28:36.159896+0000","last_fullsized":"2024-04-26T22:28:36.159896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154011+0000","last_change":"2024-04-26T22:28:36.154011+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154011+0000","last_undegraded":"2024-04-26T22:28:36.154011+0000","last_fullsized":"2024-04-26T22:28:36.154011+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170598+0000","last_change":"2024-04-26T22:28:36.165408+0000","last_active":"2024-04-26T22:28:36.170598+0000","last_peered":"2024-04-26T22:28:36.170598+0000","last_clean":"2024-04-26T22:28:36.170598+0000","last_became_active":"2024-04-26T22:28:36.165110+0000","last_became_peered":"2024-04-26T22:28:36.165110+0000","last_unstale":"2024-04-26T22:28:36.170598+0000","last_undegraded":"2024-04-26T22:28:36.170598+0000","last_fullsized":"2024-04-26T22:28:36.170598+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148308+0000","last_change":"2024-04-26T22:28:36.148308+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148308+0000","last_undegraded":"2024-04-26T22:28:36.148308+0000","last_fullsized":"2024-04-26T22:28:36.148308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.156144+0000","last_change":"2024-04-26T22:28:36.154806+0000","last_active":"2024-04-26T22:28:36.156144+0000","last_peered":"2024-04-26T22:28:36.156144+0000","last_clean":"2024-04-26T22:28:36.156144+0000","last_became_active":"2024-04-26T22:28:36.154497+0000","last_became_peered":"2024-04-26T22:28:36.154497+0000","last_unstale":"2024-04-26T22:28:36.156144+0000","last_undegraded":"2024-04-26T22:28:36.156144+0000","last_fullsized":"2024-04-26T22:28:36.156144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148250+0000","last_change":"2024-04-26T22:28:36.148250+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148250+0000","last_undegraded":"2024-04-26T22:28:36.148250+0000","last_fullsized":"2024-04-26T22:28:36.148250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170452+0000","last_change":"2024-04-26T22:28:36.165313+0000","last_active":"2024-04-26T22:28:36.170452+0000","last_peered":"2024-04-26T22:28:36.170452+0000","last_clean":"2024-04-26T22:28:36.170452+0000","last_became_active":"2024-04-26T22:28:36.165008+0000","last_became_peered":"2024-04-26T22:28:36.165008+0000","last_unstale":"2024-04-26T22:28:36.170452+0000","last_undegraded":"2024-04-26T22:28:36.170452+0000","last_fullsized":"2024-04-26T22:28:36.170452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:56.424170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157367+0000","last_change":"2024-04-26T22:28:36.157367+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157367+0000","last_undegraded":"2024-04-26T22:28:36.157367+0000","last_fullsized":"2024-04-26T22:28:36.157367+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159950+0000","last_change":"2024-04-26T22:28:36.157297+0000","last_active":"2024-04-26T22:28:36.159950+0000","last_peered":"2024-04-26T22:28:36.159950+0000","last_clean":"2024-04-26T22:28:36.159950+0000","last_became_active":"2024-04-26T22:28:36.157002+0000","last_became_peered":"2024-04-26T22:28:36.157002+0000","last_unstale":"2024-04-26T22:28:36.159950+0000","last_undegraded":"2024-04-26T22:28:36.159950+0000","last_fullsized":"2024-04-26T22:28:36.159950+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:36.488188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155983+0000","last_change":"2024-04-26T22:28:36.155983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155983+0000","last_undegraded":"2024-04-26T22:28:36.155983+0000","last_fullsized":"2024-04-26T22:28:36.155983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.172286+0000","last_change":"2024-04-26T22:28:36.167783+0000","last_active":"2024-04-26T22:28:36.172286+0000","last_peered":"2024-04-26T22:28:36.172286+0000","last_clean":"2024-04-26T22:28:36.172286+0000","last_became_active":"2024-04-26T22:28:36.167480+0000","last_became_peered":"2024-04-26T22:28:36.167480+0000","last_unstale":"2024-04-26T22:28:36.172286+0000","last_undegraded":"2024-04-26T22:28:36.172286+0000","last_fullsized":"2024-04-26T22:28:36.172286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152072+0000","last_change":"2024-04-26T22:28:36.152072+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152072+0000","last_undegraded":"2024-04-26T22:28:36.152072+0000","last_fullsized":"2024-04-26T22:28:36.152072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167154+0000","last_change":"2024-04-26T22:28:36.161210+0000","last_active":"2024-04-26T22:28:36.167154+0000","last_peered":"2024-04-26T22:28:36.167154+0000","last_clean":"2024-04-26T22:28:36.167154+0000","last_became_active":"2024-04-26T22:28:36.160683+0000","last_became_peered":"2024-04-26T22:28:36.160683+0000","last_unstale":"2024-04-26T22:28:36.167154+0000","last_undegraded":"2024-04-26T22:28:36.167154+0000","last_fullsized":"2024-04-26T22:28:36.167154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153574+0000","last_change":"2024-04-26T22:28:36.153574+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153574+0000","last_undegraded":"2024-04-26T22:28:36.153574+0000","last_fullsized":"2024-04-26T22:28:36.153574+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164527+0000","last_change":"2024-04-26T22:28:36.162766+0000","last_active":"2024-04-26T22:28:36.164527+0000","last_peered":"2024-04-26T22:28:36.164527+0000","last_clean":"2024-04-26T22:28:36.164527+0000","last_became_active":"2024-04-26T22:28:36.162468+0000","last_became_peered":"2024-04-26T22:28:36.162468+0000","last_unstale":"2024-04-26T22:28:36.164527+0000","last_undegraded":"2024-04-26T22:28:36.164527+0000","last_fullsized":"2024-04-26T22:28:36.164527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158533+0000","last_change":"2024-04-26T22:28:36.158533+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158533+0000","last_undegraded":"2024-04-26T22:28:36.158533+0000","last_fullsized":"2024-04-26T22:28:36.158533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164922+0000","last_change":"2024-04-26T22:28:36.162336+0000","last_active":"2024-04-26T22:28:36.164922+0000","last_peered":"2024-04-26T22:28:36.164922+0000","last_clean":"2024-04-26T22:28:36.164922+0000","last_became_active":"2024-04-26T22:28:36.162067+0000","last_became_peered":"2024-04-26T22:28:36.162067+0000","last_unstale":"2024-04-26T22:28:36.164922+0000","last_undegraded":"2024-04-26T22:28:36.164922+0000","last_fullsized":"2024-04-26T22:28:36.164922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148121+0000","last_change":"2024-04-26T22:28:36.148121+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148121+0000","last_undegraded":"2024-04-26T22:28:36.148121+0000","last_fullsized":"2024-04-26T22:28:36.148121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164607+0000","last_change":"2024-04-26T22:28:36.161738+0000","last_active":"2024-04-26T22:28:36.164607+0000","last_peered":"2024-04-26T22:28:36.164607+0000","last_clean":"2024-04-26T22:28:36.164607+0000","last_became_active":"2024-04-26T22:28:36.161489+0000","last_became_peered":"2024-04-26T22:28:36.161489+0000","last_unstale":"2024-04-26T22:28:36.164607+0000","last_undegraded":"2024-04-26T22:28:36.164607+0000","last_fullsized":"2024-04-26T22:28:36.164607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150196+0000","last_change":"2024-04-26T22:28:36.150196+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150196+0000","last_undegraded":"2024-04-26T22:28:36.150196+0000","last_fullsized":"2024-04-26T22:28:36.150196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164669+0000","last_change":"2024-04-26T22:28:36.159715+0000","last_active":"2024-04-26T22:28:36.164669+0000","last_peered":"2024-04-26T22:28:36.164669+0000","last_clean":"2024-04-26T22:28:36.164669+0000","last_became_active":"2024-04-26T22:28:36.159437+0000","last_became_peered":"2024-04-26T22:28:36.159437+0000","last_unstale":"2024-04-26T22:28:36.164669+0000","last_undegraded":"2024-04-26T22:28:36.164669+0000","last_fullsized":"2024-04-26T22:28:36.164669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149529+0000","last_change":"2024-04-26T22:28:36.149529+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149529+0000","last_undegraded":"2024-04-26T22:28:36.149529+0000","last_fullsized":"2024-04-26T22:28:36.149529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165270+0000","last_change":"2024-04-26T22:28:36.163369+0000","last_active":"2024-04-26T22:28:36.165270+0000","last_peered":"2024-04-26T22:28:36.165270+0000","last_clean":"2024-04-26T22:28:36.165270+0000","last_became_active":"2024-04-26T22:28:36.163060+0000","last_became_peered":"2024-04-26T22:28:36.163060+0000","last_unstale":"2024-04-26T22:28:36.165270+0000","last_undegraded":"2024-04-26T22:28:36.165270+0000","last_fullsized":"2024-04-26T22:28:36.165270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:58.196234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151707+0000","last_change":"2024-04-26T22:28:36.151707+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151707+0000","last_undegraded":"2024-04-26T22:28:36.151707+0000","last_fullsized":"2024-04-26T22:28:36.151707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.166646+0000","last_change":"2024-04-26T22:28:36.165081+0000","last_active":"2024-04-26T22:28:36.166646+0000","last_peered":"2024-04-26T22:28:36.166646+0000","last_clean":"2024-04-26T22:28:36.166646+0000","last_became_active":"2024-04-26T22:28:36.164792+0000","last_became_peered":"2024-04-26T22:28:36.164792+0000","last_unstale":"2024-04-26T22:28:36.166646+0000","last_undegraded":"2024-04-26T22:28:36.166646+0000","last_fullsized":"2024-04-26T22:28:36.166646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:54.680123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155483+0000","last_change":"2024-04-26T22:28:36.155483+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155483+0000","last_undegraded":"2024-04-26T22:28:36.155483+0000","last_fullsized":"2024-04-26T22:28:36.155483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162204+0000","last_change":"2024-04-26T22:28:36.158147+0000","last_active":"2024-04-26T22:28:36.162204+0000","last_peered":"2024-04-26T22:28:36.162204+0000","last_clean":"2024-04-26T22:28:36.162204+0000","last_became_active":"2024-04-26T22:28:36.157890+0000","last_became_peered":"2024-04-26T22:28:36.157890+0000","last_unstale":"2024-04-26T22:28:36.162204+0000","last_undegraded":"2024-04-26T22:28:36.162204+0000","last_fullsized":"2024-04-26T22:28:36.162204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151703+0000","last_change":"2024-04-26T22:28:36.151703+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151703+0000","last_undegraded":"2024-04-26T22:28:36.151703+0000","last_fullsized":"2024-04-26T22:28:36.151703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165776+0000","last_change":"2024-04-26T22:28:36.164416+0000","last_active":"2024-04-26T22:28:36.165776+0000","last_peered":"2024-04-26T22:28:36.165776+0000","last_clean":"2024-04-26T22:28:36.165776+0000","last_became_active":"2024-04-26T22:28:36.164074+0000","last_became_peered":"2024-04-26T22:28:36.164074+0000","last_unstale":"2024-04-26T22:28:36.165776+0000","last_undegraded":"2024-04-26T22:28:36.165776+0000","last_fullsized":"2024-04-26T22:28:36.165776+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:44.778533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148440+0000","last_change":"2024-04-26T22:28:36.148440+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148440+0000","last_undegraded":"2024-04-26T22:28:36.148440+0000","last_fullsized":"2024-04-26T22:28:36.148440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.168241+0000","last_change":"2024-04-26T22:28:36.167663+0000","last_active":"2024-04-26T22:28:36.168241+0000","last_peered":"2024-04-26T22:28:36.168241+0000","last_clean":"2024-04-26T22:28:36.168241+0000","last_became_active":"2024-04-26T22:28:36.167430+0000","last_became_peered":"2024-04-26T22:28:36.167430+0000","last_unstale":"2024-04-26T22:28:36.168241+0000","last_undegraded":"2024-04-26T22:28:36.168241+0000","last_fullsized":"2024-04-26T22:28:36.168241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146555+0000","last_change":"2024-04-26T22:28:36.146555+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146555+0000","last_undegraded":"2024-04-26T22:28:36.146555+0000","last_fullsized":"2024-04-26T22:28:36.146555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155478+0000","last_change":"2024-04-26T22:28:36.152997+0000","last_active":"2024-04-26T22:28:36.155478+0000","last_peered":"2024-04-26T22:28:36.155478+0000","last_clean":"2024-04-26T22:28:36.155478+0000","last_became_active":"2024-04-26T22:28:36.152694+0000","last_became_peered":"2024-04-26T22:28:36.152694+0000","last_unstale":"2024-04-26T22:28:36.155478+0000","last_undegraded":"2024-04-26T22:28:36.155478+0000","last_fullsized":"2024-04-26T22:28:36.155478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:42.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153502+0000","last_change":"2024-04-26T22:28:36.153502+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153502+0000","last_undegraded":"2024-04-26T22:28:36.153502+0000","last_fullsized":"2024-04-26T22:28:36.153502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164982+0000","last_change":"2024-04-26T22:28:36.162940+0000","last_active":"2024-04-26T22:28:36.164982+0000","last_peered":"2024-04-26T22:28:36.164982+0000","last_clean":"2024-04-26T22:28:36.164982+0000","last_became_active":"2024-04-26T22:28:36.162676+0000","last_became_peered":"2024-04-26T22:28:36.162676+0000","last_unstale":"2024-04-26T22:28:36.164982+0000","last_undegraded":"2024-04-26T22:28:36.164982+0000","last_fullsized":"2024-04-26T22:28:36.164982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153988+0000","last_change":"2024-04-26T22:28:36.153988+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153988+0000","last_undegraded":"2024-04-26T22:28:36.153988+0000","last_fullsized":"2024-04-26T22:28:36.153988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167075+0000","last_change":"2024-04-26T22:28:36.157167+0000","last_active":"2024-04-26T22:28:36.167075+0000","last_peered":"2024-04-26T22:28:36.167075+0000","last_clean":"2024-04-26T22:28:36.167075+0000","last_became_active":"2024-04-26T22:28:36.156867+0000","last_became_peered":"2024-04-26T22:28:36.156867+0000","last_unstale":"2024-04-26T22:28:36.167075+0000","last_undegraded":"2024-04-26T22:28:36.167075+0000","last_fullsized":"2024-04-26T22:28:36.167075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154215+0000","last_change":"2024-04-26T22:28:36.154215+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154215+0000","last_undegraded":"2024-04-26T22:28:36.154215+0000","last_fullsized":"2024-04-26T22:28:36.154215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.163612+0000","last_change":"2024-04-26T22:28:36.160489+0000","last_active":"2024-04-26T22:28:36.163612+0000","last_peered":"2024-04-26T22:28:36.163612+0000","last_clean":"2024-04-26T22:28:36.163612+0000","last_became_active":"2024-04-26T22:28:36.160227+0000","last_became_peered":"2024-04-26T22:28:36.160227+0000","last_unstale":"2024-04-26T22:28:36.163612+0000","last_undegraded":"2024-04-26T22:28:36.163612+0000","last_fullsized":"2024-04-26T22:28:36.163612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:23.581702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149391+0000","last_change":"2024-04-26T22:28:36.149391+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149391+0000","last_undegraded":"2024-04-26T22:28:36.149391+0000","last_fullsized":"2024-04-26T22:28:36.149391+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159929+0000","last_change":"2024-04-26T22:28:36.156431+0000","last_active":"2024-04-26T22:28:36.159929+0000","last_peered":"2024-04-26T22:28:36.159929+0000","last_clean":"2024-04-26T22:28:36.159929+0000","last_became_active":"2024-04-26T22:28:36.156129+0000","last_became_peered":"2024-04-26T22:28:36.156129+0000","last_unstale":"2024-04-26T22:28:36.159929+0000","last_undegraded":"2024-04-26T22:28:36.159929+0000","last_fullsized":"2024-04-26T22:28:36.159929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:07.246183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146638+0000","last_change":"2024-04-26T22:28:36.146638+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146638+0000","last_undegraded":"2024-04-26T22:28:36.146638+0000","last_fullsized":"2024-04-26T22:28:36.146638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167053+0000","last_change":"2024-04-26T22:28:36.156716+0000","last_active":"2024-04-26T22:28:36.167053+0000","last_peered":"2024-04-26T22:28:36.167053+0000","last_clean":"2024-04-26T22:28:36.167053+0000","last_became_active":"2024-04-26T22:28:36.156424+0000","last_became_peered":"2024-04-26T22:28:36.156424+0000","last_unstale":"2024-04-26T22:28:36.167053+0000","last_undegraded":"2024-04-26T22:28:36.167053+0000","last_fullsized":"2024-04-26T22:28:36.167053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146530+0000","last_change":"2024-04-26T22:28:36.146530+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146530+0000","last_undegraded":"2024-04-26T22:28:36.146530+0000","last_fullsized":"2024-04-26T22:28:36.146530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.160043+0000","last_change":"2024-04-26T22:28:36.157532+0000","last_active":"2024-04-26T22:28:36.160043+0000","last_peered":"2024-04-26T22:28:36.160043+0000","last_clean":"2024-04-26T22:28:36.160043+0000","last_became_active":"2024-04-26T22:28:36.157238+0000","last_became_peered":"2024-04-26T22:28:36.157238+0000","last_unstale":"2024-04-26T22:28:36.160043+0000","last_undegraded":"2024-04-26T22:28:36.160043+0000","last_fullsized":"2024-04-26T22:28:36.160043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:16.612120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151105+0000","last_change":"2024-04-26T22:28:36.151105+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151105+0000","last_undegraded":"2024-04-26T22:28:36.151105+0000","last_fullsized":"2024-04-26T22:28:36.151105+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162295+0000","last_change":"2024-04-26T22:28:36.157860+0000","last_active":"2024-04-26T22:28:36.162295+0000","last_peered":"2024-04-26T22:28:36.162295+0000","last_clean":"2024-04-26T22:28:36.162295+0000","last_became_active":"2024-04-26T22:28:36.157588+0000","last_became_peered":"2024-04-26T22:28:36.157588+0000","last_unstale":"2024-04-26T22:28:36.162295+0000","last_undegraded":"2024-04-26T22:28:36.162295+0000","last_fullsized":"2024-04-26T22:28:36.162295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153545+0000","last_change":"2024-04-26T22:28:36.153545+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153545+0000","last_undegraded":"2024-04-26T22:28:36.153545+0000","last_fullsized":"2024-04-26T22:28:36.153545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167355+0000","last_change":"2024-04-26T22:28:36.165811+0000","last_active":"2024-04-26T22:28:36.167355+0000","last_peered":"2024-04-26T22:28:36.167355+0000","last_clean":"2024-04-26T22:28:36.167355+0000","last_became_active":"2024-04-26T22:28:36.165461+0000","last_became_peered":"2024-04-26T22:28:36.165461+0000","last_unstale":"2024-04-26T22:28:36.167355+0000","last_undegraded":"2024-04-26T22:28:36.167355+0000","last_fullsized":"2024-04-26T22:28:36.167355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:18.602353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153458+0000","last_change":"2024-04-26T22:28:36.153458+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153458+0000","last_undegraded":"2024-04-26T22:28:36.153458+0000","last_fullsized":"2024-04-26T22:28:36.153458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164692+0000","last_change":"2024-04-26T22:28:36.159775+0000","last_active":"2024-04-26T22:28:36.164692+0000","last_peered":"2024-04-26T22:28:36.164692+0000","last_clean":"2024-04-26T22:28:36.164692+0000","last_became_active":"2024-04-26T22:28:36.159490+0000","last_became_peered":"2024-04-26T22:28:36.159490+0000","last_unstale":"2024-04-26T22:28:36.164692+0000","last_undegraded":"2024-04-26T22:28:36.164692+0000","last_fullsized":"2024-04-26T22:28:36.164692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149079+0000","last_change":"2024-04-26T22:28:36.149079+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149079+0000","last_undegraded":"2024-04-26T22:28:36.149079+0000","last_fullsized":"2024-04-26T22:28:36.149079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155941+0000","last_change":"2024-04-26T22:28:36.154743+0000","last_active":"2024-04-26T22:28:36.155941+0000","last_peered":"2024-04-26T22:28:36.155941+0000","last_clean":"2024-04-26T22:28:36.155941+0000","last_became_active":"2024-04-26T22:28:36.154433+0000","last_became_peered":"2024-04-26T22:28:36.154433+0000","last_unstale":"2024-04-26T22:28:36.155941+0000","last_undegraded":"2024-04-26T22:28:36.155941+0000","last_fullsized":"2024-04-26T22:28:36.155941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146336+0000","last_change":"2024-04-26T22:28:36.146336+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146336+0000","last_undegraded":"2024-04-26T22:28:36.146336+0000","last_fullsized":"2024-04-26T22:28:36.146336+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164732+0000","last_change":"2024-04-26T22:28:36.160465+0000","last_active":"2024-04-26T22:28:36.164732+0000","last_peered":"2024-04-26T22:28:36.164732+0000","last_clean":"2024-04-26T22:28:36.164732+0000","last_became_active":"2024-04-26T22:28:36.160206+0000","last_became_peered":"2024-04-26T22:28:36.160206+0000","last_unstale":"2024-04-26T22:28:36.164732+0000","last_undegraded":"2024-04-26T22:28:36.164732+0000","last_fullsized":"2024-04-26T22:28:36.164732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:47.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146383+0000","last_change":"2024-04-26T22:28:36.146383+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146383+0000","last_undegraded":"2024-04-26T22:28:36.146383+0000","last_fullsized":"2024-04-26T22:28:36.146383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164966+0000","last_change":"2024-04-26T22:28:36.163006+0000","last_active":"2024-04-26T22:28:36.164966+0000","last_peered":"2024-04-26T22:28:36.164966+0000","last_clean":"2024-04-26T22:28:36.164966+0000","last_became_active":"2024-04-26T22:28:36.162735+0000","last_became_peered":"2024-04-26T22:28:36.162735+0000","last_unstale":"2024-04-26T22:28:36.164966+0000","last_undegraded":"2024-04-26T22:28:36.164966+0000","last_fullsized":"2024-04-26T22:28:36.164966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147713+0000","last_change":"2024-04-26T22:28:36.147713+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147713+0000","last_undegraded":"2024-04-26T22:28:36.147713+0000","last_fullsized":"2024-04-26T22:28:36.147713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.161202+0000","last_change":"2024-04-26T22:28:36.159382+0000","last_active":"2024-04-26T22:28:36.161202+0000","last_peered":"2024-04-26T22:28:36.161202+0000","last_clean":"2024-04-26T22:28:36.161202+0000","last_became_active":"2024-04-26T22:28:36.159016+0000","last_became_peered":"2024-04-26T22:28:36.159016+0000","last_unstale":"2024-04-26T22:28:36.161202+0000","last_undegraded":"2024-04-26T22:28:36.161202+0000","last_fullsized":"2024-04-26T22:28:36.161202+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156626+0000","last_change":"2024-04-26T22:28:36.156626+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156626+0000","last_undegraded":"2024-04-26T22:28:36.156626+0000","last_fullsized":"2024-04-26T22:28:36.156626+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164507+0000","last_change":"2024-04-26T22:28:36.162470+0000","last_active":"2024-04-26T22:28:36.164507+0000","last_peered":"2024-04-26T22:28:36.164507+0000","last_clean":"2024-04-26T22:28:36.164507+0000","last_became_active":"2024-04-26T22:28:36.162164+0000","last_became_peered":"2024-04-26T22:28:36.162164+0000","last_unstale":"2024-04-26T22:28:36.164507+0000","last_undegraded":"2024-04-26T22:28:36.164507+0000","last_fullsized":"2024-04-26T22:28:36.164507+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155799+0000","last_change":"2024-04-26T22:28:36.155799+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155799+0000","last_undegraded":"2024-04-26T22:28:36.155799+0000","last_fullsized":"2024-04-26T22:28:36.155799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167245+0000","last_change":"2024-04-26T22:28:36.162279+0000","last_active":"2024-04-26T22:28:36.167245+0000","last_peered":"2024-04-26T22:28:36.167245+0000","last_clean":"2024-04-26T22:28:36.167245+0000","last_became_active":"2024-04-26T22:28:36.161730+0000","last_became_peered":"2024-04-26T22:28:36.161730+0000","last_unstale":"2024-04-26T22:28:36.167245+0000","last_undegraded":"2024-04-26T22:28:36.167245+0000","last_fullsized":"2024-04-26T22:28:36.167245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.152298+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:36.152298+0000","last_peered":"2024-04-26T22:28:36.152298+0000","last_clean":"2024-04-26T22:28:36.152298+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:36.152298+0000","last_undegraded":"2024-04-26T22:28:36.152298+0000","last_fullsized":"2024-04-26T22:28:36.152298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":774224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-26T22:28:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148975+0000', 'last_change': '2024-04-26T22:28:36.148975+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148975+0000', 'last_undegraded': '2024-04-26T22:28:36.148975+0000', 'last_fullsized': '2024-04-26T22:28:36.148975+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-26T22:28:40.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148634+0000', 'last_change': '2024-04-26T22:28:36.148634+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148634+0000', 'last_undegraded': '2024-04-26T22:28:36.148634+0000', 'last_fullsized': '2024-04-26T22:28:36.148634+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157768+0000', 'last_change': '2024-04-26T22:28:36.157768+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157768+0000', 'last_undegraded': '2024-04-26T22:28:36.157768+0000', 'last_fullsized': '2024-04-26T22:28:36.157768+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157943+0000', 'last_change': '2024-04-26T22:28:36.157943+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157943+0000', 'last_undegraded': '2024-04-26T22:28:36.157943+0000', 'last_fullsized': '2024-04-26T22:28:36.157943+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148758+0000', 'last_change': '2024-04-26T22:28:36.148758+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148758+0000', 'last_undegraded': '2024-04-26T22:28:36.148758+0000', 'last_fullsized': '2024-04-26T22:28:36.148758+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151939+0000', 'last_change': '2024-04-26T22:28:36.151939+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151939+0000', 'last_undegraded': '2024-04-26T22:28:36.151939+0000', 'last_fullsized': '2024-04-26T22:28:36.151939+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-26T22:28:40.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150312+0000', 'last_change': '2024-04-26T22:28:36.150312+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150312+0000', 'last_undegraded': '2024-04-26T22:28:36.150312+0000', 'last_fullsized': '2024-04-26T22:28:36.150312+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157416+0000', 'last_change': '2024-04-26T22:28:36.157416+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157416+0000', 'last_undegraded': '2024-04-26T22:28:36.157416+0000', 'last_fullsized': '2024-04-26T22:28:36.157416+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157243+0000', 'last_change': '2024-04-26T22:28:36.157243+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157243+0000', 'last_undegraded': '2024-04-26T22:28:36.157243+0000', 'last_fullsized': '2024-04-26T22:28:36.157243+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.156453+0000', 'last_change': '2024-04-26T22:28:36.156453+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.156453+0000', 'last_undegraded': '2024-04-26T22:28:36.156453+0000', 'last_fullsized': '2024-04-26T22:28:36.156453+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154387+0000', 'last_change': '2024-04-26T22:28:36.154387+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154387+0000', 'last_undegraded': '2024-04-26T22:28:36.154387+0000', 'last_fullsized': '2024-04-26T22:28:36.154387+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150951+0000', 'last_change': '2024-04-26T22:28:36.150951+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150951+0000', 'last_undegraded': '2024-04-26T22:28:36.150951+0000', 'last_fullsized': '2024-04-26T22:28:36.150951+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.147261+0000', 'last_change': '2024-04-26T22:28:36.147261+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.147261+0000', 'last_undegraded': '2024-04-26T22:28:36.147261+0000', 'last_fullsized': '2024-04-26T22:28:36.147261+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154697+0000', 'last_change': '2024-04-26T22:28:36.154697+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154697+0000', 'last_undegraded': '2024-04-26T22:28:36.154697+0000', 'last_fullsized': '2024-04-26T22:28:36.154697+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.152160+0000', 'last_change': '2024-04-26T22:28:36.152160+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.152160+0000', 'last_undegraded': '2024-04-26T22:28:36.152160+0000', 'last_fullsized': '2024-04-26T22:28:36.152160+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151319+0000', 'last_change': '2024-04-26T22:28:36.151319+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151319+0000', 'last_undegraded': '2024-04-26T22:28:36.151319+0000', 'last_fullsized': '2024-04-26T22:28:36.151319+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157224+0000', 'last_change': '2024-04-26T22:28:36.157224+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157224+0000', 'last_undegraded': '2024-04-26T22:28:36.157224+0000', 'last_fullsized': '2024-04-26T22:28:36.157224+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-26T22:28:40.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.159081+0000', 'last_change': '2024-04-26T22:28:36.159081+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.159081+0000', 'last_undegraded': '2024-04-26T22:28:36.159081+0000', 'last_fullsized': '2024-04-26T22:28:36.159081+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.152582+0000', 'last_change': '2024-04-26T22:28:36.152582+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.152582+0000', 'last_undegraded': '2024-04-26T22:28:36.152582+0000', 'last_fullsized': '2024-04-26T22:28:36.152582+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148009+0000', 'last_change': '2024-04-26T22:28:36.148009+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148009+0000', 'last_undegraded': '2024-04-26T22:28:36.148009+0000', 'last_fullsized': '2024-04-26T22:28:36.148009+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150983+0000', 'last_change': '2024-04-26T22:28:36.150983+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150983+0000', 'last_undegraded': '2024-04-26T22:28:36.150983+0000', 'last_fullsized': '2024-04-26T22:28:36.150983+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154224+0000', 'last_change': '2024-04-26T22:28:36.154224+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154224+0000', 'last_undegraded': '2024-04-26T22:28:36.154224+0000', 'last_fullsized': '2024-04-26T22:28:36.154224+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.152240+0000', 'last_change': '2024-04-26T22:28:36.152240+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.152240+0000', 'last_undegraded': '2024-04-26T22:28:36.152240+0000', 'last_fullsized': '2024-04-26T22:28:36.152240+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-26T22:28:40.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150753+0000', 'last_change': '2024-04-26T22:28:36.150753+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150753+0000', 'last_undegraded': '2024-04-26T22:28:36.150753+0000', 'last_fullsized': '2024-04-26T22:28:36.150753+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154621+0000', 'last_change': '2024-04-26T22:28:36.154621+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154621+0000', 'last_undegraded': '2024-04-26T22:28:36.154621+0000', 'last_fullsized': '2024-04-26T22:28:36.154621+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.147145+0000', 'last_change': '2024-04-26T22:28:36.147145+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.147145+0000', 'last_undegraded': '2024-04-26T22:28:36.147145+0000', 'last_fullsized': '2024-04-26T22:28:36.147145+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157292+0000', 'last_change': '2024-04-26T22:28:36.157292+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157292+0000', 'last_undegraded': '2024-04-26T22:28:36.157292+0000', 'last_fullsized': '2024-04-26T22:28:36.157292+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150520+0000', 'last_change': '2024-04-26T22:28:36.150520+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150520+0000', 'last_undegraded': '2024-04-26T22:28:36.150520+0000', 'last_fullsized': '2024-04-26T22:28:36.150520+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-26T22:28:40.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151361+0000', 'last_change': '2024-04-26T22:28:36.151361+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151361+0000', 'last_undegraded': '2024-04-26T22:28:36.151361+0000', 'last_fullsized': '2024-04-26T22:28:36.151361+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.158720+0000', 'last_change': '2024-04-26T22:28:36.158720+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.158720+0000', 'last_undegraded': '2024-04-26T22:28:36.158720+0000', 'last_fullsized': '2024-04-26T22:28:36.158720+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150207+0000', 'last_change': '2024-04-26T22:28:36.150207+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150207+0000', 'last_undegraded': '2024-04-26T22:28:36.150207+0000', 'last_fullsized': '2024-04-26T22:28:36.150207+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.147125+0000', 'last_change': '2024-04-26T22:28:36.147125+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.147125+0000', 'last_undegraded': '2024-04-26T22:28:36.147125+0000', 'last_fullsized': '2024-04-26T22:28:36.147125+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.147145+0000', 'last_change': '2024-04-26T22:28:36.147145+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.147145+0000', 'last_undegraded': '2024-04-26T22:28:36.147145+0000', 'last_fullsized': '2024-04-26T22:28:36.147145+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154011+0000', 'last_change': '2024-04-26T22:28:36.154011+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154011+0000', 'last_undegraded': '2024-04-26T22:28:36.154011+0000', 'last_fullsized': '2024-04-26T22:28:36.154011+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-26T22:28:40.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148308+0000', 'last_change': '2024-04-26T22:28:36.148308+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148308+0000', 'last_undegraded': '2024-04-26T22:28:36.148308+0000', 'last_fullsized': '2024-04-26T22:28:36.148308+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148250+0000', 'last_change': '2024-04-26T22:28:36.148250+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148250+0000', 'last_undegraded': '2024-04-26T22:28:36.148250+0000', 'last_fullsized': '2024-04-26T22:28:36.148250+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.157367+0000', 'last_change': '2024-04-26T22:28:36.157367+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.157367+0000', 'last_undegraded': '2024-04-26T22:28:36.157367+0000', 'last_fullsized': '2024-04-26T22:28:36.157367+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.155983+0000', 'last_change': '2024-04-26T22:28:36.155983+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.155983+0000', 'last_undegraded': '2024-04-26T22:28:36.155983+0000', 'last_fullsized': '2024-04-26T22:28:36.155983+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.152072+0000', 'last_change': '2024-04-26T22:28:36.152072+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.152072+0000', 'last_undegraded': '2024-04-26T22:28:36.152072+0000', 'last_fullsized': '2024-04-26T22:28:36.152072+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.153574+0000', 'last_change': '2024-04-26T22:28:36.153574+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.153574+0000', 'last_undegraded': '2024-04-26T22:28:36.153574+0000', 'last_fullsized': '2024-04-26T22:28:36.153574+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.158533+0000', 'last_change': '2024-04-26T22:28:36.158533+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.158533+0000', 'last_undegraded': '2024-04-26T22:28:36.158533+0000', 'last_fullsized': '2024-04-26T22:28:36.158533+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148121+0000', 'last_change': '2024-04-26T22:28:36.148121+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148121+0000', 'last_undegraded': '2024-04-26T22:28:36.148121+0000', 'last_fullsized': '2024-04-26T22:28:36.148121+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.150196+0000', 'last_change': '2024-04-26T22:28:36.150196+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.150196+0000', 'last_undegraded': '2024-04-26T22:28:36.150196+0000', 'last_fullsized': '2024-04-26T22:28:36.150196+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.149529+0000', 'last_change': '2024-04-26T22:28:36.149529+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.149529+0000', 'last_undegraded': '2024-04-26T22:28:36.149529+0000', 'last_fullsized': '2024-04-26T22:28:36.149529+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151707+0000', 'last_change': '2024-04-26T22:28:36.151707+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151707+0000', 'last_undegraded': '2024-04-26T22:28:36.151707+0000', 'last_fullsized': '2024-04-26T22:28:36.151707+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-26T22:28:40.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.155483+0000', 'last_change': '2024-04-26T22:28:36.155483+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.155483+0000', 'last_undegraded': '2024-04-26T22:28:36.155483+0000', 'last_fullsized': '2024-04-26T22:28:36.155483+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151703+0000', 'last_change': '2024-04-26T22:28:36.151703+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151703+0000', 'last_undegraded': '2024-04-26T22:28:36.151703+0000', 'last_fullsized': '2024-04-26T22:28:36.151703+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.148440+0000', 'last_change': '2024-04-26T22:28:36.148440+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.148440+0000', 'last_undegraded': '2024-04-26T22:28:36.148440+0000', 'last_fullsized': '2024-04-26T22:28:36.148440+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.146555+0000', 'last_change': '2024-04-26T22:28:36.146555+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.146555+0000', 'last_undegraded': '2024-04-26T22:28:36.146555+0000', 'last_fullsized': '2024-04-26T22:28:36.146555+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.153502+0000', 'last_change': '2024-04-26T22:28:36.153502+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.153502+0000', 'last_undegraded': '2024-04-26T22:28:36.153502+0000', 'last_fullsized': '2024-04-26T22:28:36.153502+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-26T22:28:40.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.153988+0000', 'last_change': '2024-04-26T22:28:36.153988+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.153988+0000', 'last_undegraded': '2024-04-26T22:28:36.153988+0000', 'last_fullsized': '2024-04-26T22:28:36.153988+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.154215+0000', 'last_change': '2024-04-26T22:28:36.154215+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.154215+0000', 'last_undegraded': '2024-04-26T22:28:36.154215+0000', 'last_fullsized': '2024-04-26T22:28:36.154215+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.149391+0000', 'last_change': '2024-04-26T22:28:36.149391+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.149391+0000', 'last_undegraded': '2024-04-26T22:28:36.149391+0000', 'last_fullsized': '2024-04-26T22:28:36.149391+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.146638+0000', 'last_change': '2024-04-26T22:28:36.146638+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.146638+0000', 'last_undegraded': '2024-04-26T22:28:36.146638+0000', 'last_fullsized': '2024-04-26T22:28:36.146638+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.146530+0000', 'last_change': '2024-04-26T22:28:36.146530+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.146530+0000', 'last_undegraded': '2024-04-26T22:28:36.146530+0000', 'last_fullsized': '2024-04-26T22:28:36.146530+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-26T22:28:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.151105+0000', 'last_change': '2024-04-26T22:28:36.151105+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.151105+0000', 'last_undegraded': '2024-04-26T22:28:36.151105+0000', 'last_fullsized': '2024-04-26T22:28:36.151105+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.153545+0000', 'last_change': '2024-04-26T22:28:36.153545+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.153545+0000', 'last_undegraded': '2024-04-26T22:28:36.153545+0000', 'last_fullsized': '2024-04-26T22:28:36.153545+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.153458+0000', 'last_change': '2024-04-26T22:28:36.153458+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.153458+0000', 'last_undegraded': '2024-04-26T22:28:36.153458+0000', 'last_fullsized': '2024-04-26T22:28:36.153458+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.149079+0000', 'last_change': '2024-04-26T22:28:36.149079+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.149079+0000', 'last_undegraded': '2024-04-26T22:28:36.149079+0000', 'last_fullsized': '2024-04-26T22:28:36.149079+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.146336+0000', 'last_change': '2024-04-26T22:28:36.146336+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.146336+0000', 'last_undegraded': '2024-04-26T22:28:36.146336+0000', 'last_fullsized': '2024-04-26T22:28:36.146336+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.146383+0000', 'last_change': '2024-04-26T22:28:36.146383+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.146383+0000', 'last_undegraded': '2024-04-26T22:28:36.146383+0000', 'last_fullsized': '2024-04-26T22:28:36.146383+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.147713+0000', 'last_change': '2024-04-26T22:28:36.147713+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.147713+0000', 'last_undegraded': '2024-04-26T22:28:36.147713+0000', 'last_fullsized': '2024-04-26T22:28:36.147713+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.156626+0000', 'last_change': '2024-04-26T22:28:36.156626+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.156626+0000', 'last_undegraded': '2024-04-26T22:28:36.156626+0000', 'last_fullsized': '2024-04-26T22:28:36.156626+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-26T22:28:40.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-04-26T22:28:36.155799+0000', 'last_change': '2024-04-26T22:28:36.155799+0000', 'last_active': '2024-04-26T22:28:36.133205+0000', 'last_peered': '2024-04-26T22:28:36.133205+0000', 'last_clean': '2024-04-26T22:28:36.133205+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T22:28:36.155799+0000', 'last_undegraded': '2024-04-26T22:28:36.155799+0000', 'last_fullsized': '2024-04-26T22:28:36.155799+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'last_clean_scrub_stamp': '2024-04-26T22:28:36.133205+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T22:28:40.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T22:28:41.004 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.004+0000 7fb4df19e640 1 Processor -- start 2024-04-26T22:28:41.005 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.004+0000 7fb4df19e640 1 -- start start 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4df19e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4df19e640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d80d0b20 con 0x7fb4d80b5e90 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4df19e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d80d0e00 con 0x7fb4d80d2d50 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4df19e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d80b4680 con 0x7fb4d815f2f0 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4de99d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fb4d80b5e90 0x7fb4d80d1e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56638/0 (socket says 172.21.15.62:56638) 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4de99d640 1 -- 172.21.15.62:0/165488495 learned_addr learned my addr 172.21.15.62:0/165488495 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:41.006 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.005+0000 7fb4de19c640 1 --2- 172.21.15.62:0/165488495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64238413 0 0) 0x7fb4d80d0b20 con 0x7fb4d80b5e90 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4c0003520 con 0x7fb4d80b5e90 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488745025 0 0) 0x7fb4d80d0e00 con 0x7fb4d80d2d50 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4de19c640 1 -- 172.21.15.62:0/165488495 >> v1:172.21.15.174:6789/0 conn(0x7fb4d80d2d50 legacy=0x7fb4d80d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4de19c640 1 -- 172.21.15.62:0/165488495 >> v1:172.21.15.62:6789/0 conn(0x7fb4d80b5e90 legacy=0x7fb4d80d1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.007 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.006+0000 7fb4de19c640 1 -- 172.21.15.62:0/165488495 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d80b4960 con 0x7fb4d815f2f0 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4de19c640 1 --2- 172.21.15.62:0/165488495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80d3450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00097d0 tx=0x7fb4d002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac5f8ae2b7eaf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0037070 con 0x7fb4d815f2f0 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0032440 con 0x7fb4d815f2f0 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4dd19a640 1 -- 172.21.15.62:0/165488495 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00327f0 con 0x7fb4d815f2f0 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 -- 172.21.15.62:0/165488495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 msgr2=0x7fb4d80d3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 --2- 172.21.15.62:0/165488495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80d3450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00097d0 tx=0x7fb4d002a210 comp rx=0 tx=0).stop 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 -- 172.21.15.62:0/165488495 shutdown_connections 2024-04-26T22:28:41.008 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 --2- 172.21.15.62:0/165488495 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80d3450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 -- 172.21.15.62:0/165488495 >> 172.21.15.62:0/165488495 conn(0x7fb4d80d4f70 msgr2=0x7fb4d80d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 -- 172.21.15.62:0/165488495 shutdown_connections 2024-04-26T22:28:41.009 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.007+0000 7fb4df19e640 1 -- 172.21.15.62:0/165488495 wait complete. 2024-04-26T22:28:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.009+0000 7fb4df19e640 1 Processor -- start 2024-04-26T22:28:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.009+0000 7fb4df19e640 1 -- start start 2024-04-26T22:28:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4d80b5e90 0x7fb4d815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.011 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d815f6d0 con 0x7fb4d80d2d50 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d80d0900 con 0x7fb4d80b5e90 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d815f850 con 0x7fb4d815f2f0 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60720/0 (socket says 172.21.15.62:60720) 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 -- 172.21.15.62:0/3878397431 learned_addr learned my addr 172.21.15.62:0/3878397431 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4de19c640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4d80b5e90 0x7fb4d815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4de99d640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 -- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 msgr2=0x7fb4d80b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 -- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4d80b5e90 msgr2=0x7fb4d815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4d80b5e90 0x7fb4d815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 -- 172.21.15.62:0/3878397431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d80b85f0 con 0x7fb4d80d2d50 2024-04-26T22:28:41.012 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4de99d640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4dd99b640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c4b0 tx=0x7fb4cc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4d80d2d50 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4cc005480 con 0x7fb4d80d2d50 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d81f88f0 con 0x7fb4d80d2d50 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.010+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc003830 con 0x7fb4d80d2d50 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.011+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81f8eb0 con 0x7fb4d80d2d50 2024-04-26T22:28:41.013 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.011+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d8076890 con 0x7fb4d80d2d50 2024-04-26T22:28:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.012+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fb4cc01a070 con 0x7fb4d80d2d50 2024-04-26T22:28:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.012+0000 7fb4d6ffd640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 0x7fb4c406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.012+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb4cc08af30 con 0x7fb4d80d2d50 2024-04-26T22:28:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.013+0000 7fb4de19c640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 0x7fb4c406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.014 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.013+0000 7fb4de19c640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 0x7fb4c406ee70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0034000 tx=0x7fb4d002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.016 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.017+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4cc059370 con 0x7fb4d80d2d50 2024-04-26T22:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.274+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb4d81628b0 con 0x7fb4d80d2d50 2024-04-26T22:28:41.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.275+0000 7fb4d6ffd640 1 -- 172.21.15.62:0/3878397431 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1453 (secure 0 0 0) 0x7fb4cc05f030 con 0x7fb4d80d2d50 2024-04-26T22:28:41.275 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:41.275 INFO:teuthology.orchestra.run.smithi062.stdout:{"fsid":"f70b0a7c-519a-45f1-aa13-a1f2de1080a4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":111,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":8,"num_up_osds":8,"osd_up_since":1714170420,"num_in_osds":8,"osd_in_since":1714170411,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":226336768,"bytes_avail":772867776512,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":23,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5559}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T22:28:25.656036+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 msgr2=0x7fb4c406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 0x7fb4c406ee70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0034000 tx=0x7fb4d002e000 comp rx=0 tx=0).stop 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 msgr2=0x7fb4d80b7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c4b0 tx=0x7fb4cc00c970 comp rx=0 tx=0).stop 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 shutdown_connections 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fb4c406c9b0 0x7fb4c406ee70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb4d815f2f0 0x7fb4d80b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb4d80d2d50 0x7fb4d80b7b70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 --2- 172.21.15.62:0/3878397431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4d80b5e90 0x7fb4d815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 >> 172.21.15.62:0/3878397431 conn(0x7fb4d80d4f70 msgr2=0x7fb4d80b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 shutdown_connections 2024-04-26T22:28:41.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.277+0000 7fb4df19e640 1 -- 172.21.15.62:0/3878397431 wait complete. 2024-04-26T22:28:41.288 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T22:28:41.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:41.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.388+0000 7f36585af640 1 Processor -- start 2024-04-26T22:28:41.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.388+0000 7f36585af640 1 -- start start 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f36585af640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f365006ea40 0x7f36500d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f36585af640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36500b1d70 con 0x7f3650162e40 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f36585af640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36500d1960 con 0x7f36500cf830 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f36585af640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f3650163220 con 0x7f365006ea40 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f365634b640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f3650162e40 0x7f36500b2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:56650/0 (socket says 172.21.15.62:56650) 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f365634b640 1 -- 172.21.15.62:0/734780997 learned_addr learned my addr 172.21.15.62:0/734780997 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f3655b4a640 1 --2- 172.21.15.62:0/734780997 >> v2:172.21.15.174:3300/0 conn(0x7f365006ea40 0x7f36500d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588816456 0 0) 0x7f36500b1d70 con 0x7f3650162e40 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.389+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3638003520 con 0x7f3650162e40 2024-04-26T22:28:41.392 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 824596316 0 0) 0x7f3638003520 con 0x7f3650162e40 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 >> v2:172.21.15.174:3300/0 conn(0x7f365006ea40 msgr2=0x7f36500d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 --2- 172.21.15.62:0/734780997 >> v2:172.21.15.174:3300/0 conn(0x7f365006ea40 0x7f36500d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 >> v1:172.21.15.174:6789/0 conn(0x7f36500cf830 legacy=0x7f36500cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36500d1c40 con 0x7f3650162e40 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f3648001670 con 0x7f3650162e40 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 >> v1:172.21.15.62:6789/0 conn(0x7f3650162e40 legacy=0x7f36500b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f363800c130 con 0x7f3638004800 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f363800c2e0 con 0x7f36380039b0 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f363800c4e0 con 0x7f36380051c0 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f365634b640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3656b4c640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 -- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36380051c0 msgr2=0x7f363800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 -- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36380039b0 msgr2=0x7f3638003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36380039b0 0x7f3638003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3655b4a640 1 -- 172.21.15.62:0/734780997 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3638003520 con 0x7f3638004800 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3656b4c640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36380051c0 0x7f363800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.390+0000 7f3656b4c640 1 -- 172.21.15.62:0/734780997 reap_dead start 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f3655b4a640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 0x7f3638004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3644009ac0 tx=0x7f3638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ad2f35ad3aadb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644036070 con 0x7f3638004800 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644031400 con 0x7f3638004800 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f3655349640 1 -- 172.21.15.62:0/734780997 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644009690 con 0x7f3638004800 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f36585af640 1 -- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 msgr2=0x7f3638004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.391+0000 7f36585af640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 0x7f3638004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3644009ac0 tx=0x7f3638005830 comp rx=0 tx=0).stop 2024-04-26T22:28:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.392+0000 7f36585af640 1 -- 172.21.15.62:0/734780997 shutdown_connections 2024-04-26T22:28:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.392+0000 7f36585af640 1 --2- 172.21.15.62:0/734780997 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3638004800 0x7f3638004c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.392+0000 7f36585af640 1 -- 172.21.15.62:0/734780997 >> 172.21.15.62:0/734780997 conn(0x7f365009ba50 msgr2=0x7f365009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.392+0000 7f36585af640 1 -- 172.21.15.62:0/734780997 shutdown_connections 2024-04-26T22:28:41.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.392+0000 7f36585af640 1 -- 172.21.15.62:0/734780997 wait complete. 2024-04-26T22:28:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.393+0000 7f36585af640 1 Processor -- start 2024-04-26T22:28:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.393+0000 7f36585af640 1 -- start start 2024-04-26T22:28:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.393+0000 7f36585af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3638004800 0x7f36500c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.397 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36500c77c0 0x7f365014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f36500d1340 con 0x7f365015b790 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36500d0eb0 con 0x7f3638004800 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f36500d14c0 con 0x7f36500c77c0 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60746/0 (socket says 172.21.15.62:60746) 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 -- 172.21.15.62:0/2487008455 learned_addr learned my addr 172.21.15.62:0/2487008455 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f365634b640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3638004800 0x7f36500c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3656b4c640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36500c77c0 0x7f365014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 -- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36500c77c0 msgr2=0x7f365014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36500c77c0 0x7f365014fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 -- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3638004800 msgr2=0x7f36500c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3638004800 0x7f36500c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 -- 172.21.15.62:0/2487008455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365015db70 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f3655b4a640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3644034040 tx=0x7f3644004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364403f020 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3644037040 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36500b0800 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644036070 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.394+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365015d890 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.396+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365006a360 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.396+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f3644042020 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.396+0000 7f364effd640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 0x7f363c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.396+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f36440abc30 con 0x7f365015b790 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.396+0000 7f365634b640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 0x7f363c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:41.399 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.397+0000 7f365634b640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 0x7f363c06ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3648001eb0 tx=0x7f3648009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:41.400 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.400+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3644049030 con 0x7f365015b790 2024-04-26T22:28:41.584 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.584+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36500b4560 con 0x7f363c06c980 2024-04-26T22:28:41.590 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.590+0000 7f364effd640 1 -- 172.21.15.62:0/2487008455 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246637 (secure 0 0 0) 0x7f36500b4560 con 0x7f363c06c980 2024-04-26T22:28:41.593 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:41.594 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:41.595 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 msgr2=0x7f363c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 0x7f363c06ee40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3648001eb0 tx=0x7f3648009040 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 msgr2=0x7f36500b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3644034040 tx=0x7f3644004730 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 shutdown_connections 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f363c06c980 0x7f363c06ee40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f36500c77c0 0x7f365014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f365015b790 0x7f36500b0fc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 --2- 172.21.15.62:0/2487008455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3638004800 0x7f36500c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 >> 172.21.15.62:0/2487008455 conn(0x7f365009ba50 msgr2=0x7f36500cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 shutdown_connections 2024-04-26T22:28:41.596 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:41.593+0000 7f36585af640 1 -- 172.21.15.62:0/2487008455 wait complete. 2024-04-26T22:28:41.604 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-26T22:28:39.658448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221032,"kb_used_data":4856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753688,"statfs":{"total":773094113280,"available":772867776512,"internally_reserved":0,"allocated":4972544,"data_stored":2676652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.524630"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148975+0000","last_change":"2024-04-26T22:28:36.148975+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148975+0000","last_undegraded":"2024-04-26T22:28:36.148975+0000","last_fullsized":"2024-04-26T22:28:36.148975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148634+0000","last_change":"2024-04-26T22:28:36.148634+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148634+0000","last_undegraded":"2024-04-26T22:28:36.148634+0000","last_fullsized":"2024-04-26T22:28:36.148634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157768+0000","last_change":"2024-04-26T22:28:36.157768+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157768+0000","last_undegraded":"2024-04-26T22:28:36.157768+0000","last_fullsized":"2024-04-26T22:28:36.157768+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157943+0000","last_change":"2024-04-26T22:28:36.157943+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157943+0000","last_undegraded":"2024-04-26T22:28:36.157943+0000","last_fullsized":"2024-04-26T22:28:36.157943+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148758+0000","last_change":"2024-04-26T22:28:36.148758+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148758+0000","last_undegraded":"2024-04-26T22:28:36.148758+0000","last_fullsized":"2024-04-26T22:28:36.148758+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151939+0000","last_change":"2024-04-26T22:28:36.151939+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151939+0000","last_undegraded":"2024-04-26T22:28:36.151939+0000","last_fullsized":"2024-04-26T22:28:36.151939+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150312+0000","last_change":"2024-04-26T22:28:36.150312+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150312+0000","last_undegraded":"2024-04-26T22:28:36.150312+0000","last_fullsized":"2024-04-26T22:28:36.150312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157416+0000","last_change":"2024-04-26T22:28:36.157416+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157416+0000","last_undegraded":"2024-04-26T22:28:36.157416+0000","last_fullsized":"2024-04-26T22:28:36.157416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157243+0000","last_change":"2024-04-26T22:28:36.157243+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157243+0000","last_undegraded":"2024-04-26T22:28:36.157243+0000","last_fullsized":"2024-04-26T22:28:36.157243+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156453+0000","last_change":"2024-04-26T22:28:36.156453+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156453+0000","last_undegraded":"2024-04-26T22:28:36.156453+0000","last_fullsized":"2024-04-26T22:28:36.156453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154387+0000","last_change":"2024-04-26T22:28:36.154387+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154387+0000","last_undegraded":"2024-04-26T22:28:36.154387+0000","last_fullsized":"2024-04-26T22:28:36.154387+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150951+0000","last_change":"2024-04-26T22:28:36.150951+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150951+0000","last_undegraded":"2024-04-26T22:28:36.150951+0000","last_fullsized":"2024-04-26T22:28:36.150951+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147261+0000","last_change":"2024-04-26T22:28:36.147261+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147261+0000","last_undegraded":"2024-04-26T22:28:36.147261+0000","last_fullsized":"2024-04-26T22:28:36.147261+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154697+0000","last_change":"2024-04-26T22:28:36.154697+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154697+0000","last_undegraded":"2024-04-26T22:28:36.154697+0000","last_fullsized":"2024-04-26T22:28:36.154697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152160+0000","last_change":"2024-04-26T22:28:36.152160+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152160+0000","last_undegraded":"2024-04-26T22:28:36.152160+0000","last_fullsized":"2024-04-26T22:28:36.152160+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151319+0000","last_change":"2024-04-26T22:28:36.151319+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151319+0000","last_undegraded":"2024-04-26T22:28:36.151319+0000","last_fullsized":"2024-04-26T22:28:36.151319+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157224+0000","last_change":"2024-04-26T22:28:36.157224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157224+0000","last_undegraded":"2024-04-26T22:28:36.157224+0000","last_fullsized":"2024-04-26T22:28:36.157224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.159081+0000","last_change":"2024-04-26T22:28:36.159081+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.159081+0000","last_undegraded":"2024-04-26T22:28:36.159081+0000","last_fullsized":"2024-04-26T22:28:36.159081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152582+0000","last_change":"2024-04-26T22:28:36.152582+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152582+0000","last_undegraded":"2024-04-26T22:28:36.152582+0000","last_fullsized":"2024-04-26T22:28:36.152582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148009+0000","last_change":"2024-04-26T22:28:36.148009+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148009+0000","last_undegraded":"2024-04-26T22:28:36.148009+0000","last_fullsized":"2024-04-26T22:28:36.148009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150983+0000","last_change":"2024-04-26T22:28:36.150983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150983+0000","last_undegraded":"2024-04-26T22:28:36.150983+0000","last_fullsized":"2024-04-26T22:28:36.150983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154224+0000","last_change":"2024-04-26T22:28:36.154224+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154224+0000","last_undegraded":"2024-04-26T22:28:36.154224+0000","last_fullsized":"2024-04-26T22:28:36.154224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152240+0000","last_change":"2024-04-26T22:28:36.152240+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152240+0000","last_undegraded":"2024-04-26T22:28:36.152240+0000","last_fullsized":"2024-04-26T22:28:36.152240+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150753+0000","last_change":"2024-04-26T22:28:36.150753+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150753+0000","last_undegraded":"2024-04-26T22:28:36.150753+0000","last_fullsized":"2024-04-26T22:28:36.150753+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154621+0000","last_change":"2024-04-26T22:28:36.154621+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154621+0000","last_undegraded":"2024-04-26T22:28:36.154621+0000","last_fullsized":"2024-04-26T22:28:36.154621+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157292+0000","last_change":"2024-04-26T22:28:36.157292+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157292+0000","last_undegraded":"2024-04-26T22:28:36.157292+0000","last_fullsized":"2024-04-26T22:28:36.157292+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150520+0000","last_change":"2024-04-26T22:28:36.150520+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150520+0000","last_undegraded":"2024-04-26T22:28:36.150520+0000","last_fullsized":"2024-04-26T22:28:36.150520+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151361+0000","last_change":"2024-04-26T22:28:36.151361+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151361+0000","last_undegraded":"2024-04-26T22:28:36.151361+0000","last_fullsized":"2024-04-26T22:28:36.151361+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158720+0000","last_change":"2024-04-26T22:28:36.158720+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158720+0000","last_undegraded":"2024-04-26T22:28:36.158720+0000","last_fullsized":"2024-04-26T22:28:36.158720+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150207+0000","last_change":"2024-04-26T22:28:36.150207+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150207+0000","last_undegraded":"2024-04-26T22:28:36.150207+0000","last_fullsized":"2024-04-26T22:28:36.150207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147125+0000","last_change":"2024-04-26T22:28:36.147125+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147125+0000","last_undegraded":"2024-04-26T22:28:36.147125+0000","last_fullsized":"2024-04-26T22:28:36.147125+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147145+0000","last_change":"2024-04-26T22:28:36.147145+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147145+0000","last_undegraded":"2024-04-26T22:28:36.147145+0000","last_fullsized":"2024-04-26T22:28:36.147145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159896+0000","last_change":"2024-04-26T22:28:36.157128+0000","last_active":"2024-04-26T22:28:36.159896+0000","last_peered":"2024-04-26T22:28:36.159896+0000","last_clean":"2024-04-26T22:28:36.159896+0000","last_became_active":"2024-04-26T22:28:36.156854+0000","last_became_peered":"2024-04-26T22:28:36.156854+0000","last_unstale":"2024-04-26T22:28:36.159896+0000","last_undegraded":"2024-04-26T22:28:36.159896+0000","last_fullsized":"2024-04-26T22:28:36.159896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154011+0000","last_change":"2024-04-26T22:28:36.154011+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154011+0000","last_undegraded":"2024-04-26T22:28:36.154011+0000","last_fullsized":"2024-04-26T22:28:36.154011+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170598+0000","last_change":"2024-04-26T22:28:36.165408+0000","last_active":"2024-04-26T22:28:36.170598+0000","last_peered":"2024-04-26T22:28:36.170598+0000","last_clean":"2024-04-26T22:28:36.170598+0000","last_became_active":"2024-04-26T22:28:36.165110+0000","last_became_peered":"2024-04-26T22:28:36.165110+0000","last_unstale":"2024-04-26T22:28:36.170598+0000","last_undegraded":"2024-04-26T22:28:36.170598+0000","last_fullsized":"2024-04-26T22:28:36.170598+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148308+0000","last_change":"2024-04-26T22:28:36.148308+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148308+0000","last_undegraded":"2024-04-26T22:28:36.148308+0000","last_fullsized":"2024-04-26T22:28:36.148308+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.156144+0000","last_change":"2024-04-26T22:28:36.154806+0000","last_active":"2024-04-26T22:28:36.156144+0000","last_peered":"2024-04-26T22:28:36.156144+0000","last_clean":"2024-04-26T22:28:36.156144+0000","last_became_active":"2024-04-26T22:28:36.154497+0000","last_became_peered":"2024-04-26T22:28:36.154497+0000","last_unstale":"2024-04-26T22:28:36.156144+0000","last_undegraded":"2024-04-26T22:28:36.156144+0000","last_fullsized":"2024-04-26T22:28:36.156144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148250+0000","last_change":"2024-04-26T22:28:36.148250+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148250+0000","last_undegraded":"2024-04-26T22:28:36.148250+0000","last_fullsized":"2024-04-26T22:28:36.148250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.170452+0000","last_change":"2024-04-26T22:28:36.165313+0000","last_active":"2024-04-26T22:28:36.170452+0000","last_peered":"2024-04-26T22:28:36.170452+0000","last_clean":"2024-04-26T22:28:36.170452+0000","last_became_active":"2024-04-26T22:28:36.165008+0000","last_became_peered":"2024-04-26T22:28:36.165008+0000","last_unstale":"2024-04-26T22:28:36.170452+0000","last_undegraded":"2024-04-26T22:28:36.170452+0000","last_fullsized":"2024-04-26T22:28:36.170452+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:56.424170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.157367+0000","last_change":"2024-04-26T22:28:36.157367+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.157367+0000","last_undegraded":"2024-04-26T22:28:36.157367+0000","last_fullsized":"2024-04-26T22:28:36.157367+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159950+0000","last_change":"2024-04-26T22:28:36.157297+0000","last_active":"2024-04-26T22:28:36.159950+0000","last_peered":"2024-04-26T22:28:36.159950+0000","last_clean":"2024-04-26T22:28:36.159950+0000","last_became_active":"2024-04-26T22:28:36.157002+0000","last_became_peered":"2024-04-26T22:28:36.157002+0000","last_unstale":"2024-04-26T22:28:36.159950+0000","last_undegraded":"2024-04-26T22:28:36.159950+0000","last_fullsized":"2024-04-26T22:28:36.159950+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:36.488188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155983+0000","last_change":"2024-04-26T22:28:36.155983+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155983+0000","last_undegraded":"2024-04-26T22:28:36.155983+0000","last_fullsized":"2024-04-26T22:28:36.155983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.172286+0000","last_change":"2024-04-26T22:28:36.167783+0000","last_active":"2024-04-26T22:28:36.172286+0000","last_peered":"2024-04-26T22:28:36.172286+0000","last_clean":"2024-04-26T22:28:36.172286+0000","last_became_active":"2024-04-26T22:28:36.167480+0000","last_became_peered":"2024-04-26T22:28:36.167480+0000","last_unstale":"2024-04-26T22:28:36.172286+0000","last_undegraded":"2024-04-26T22:28:36.172286+0000","last_fullsized":"2024-04-26T22:28:36.172286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.152072+0000","last_change":"2024-04-26T22:28:36.152072+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.152072+0000","last_undegraded":"2024-04-26T22:28:36.152072+0000","last_fullsized":"2024-04-26T22:28:36.152072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167154+0000","last_change":"2024-04-26T22:28:36.161210+0000","last_active":"2024-04-26T22:28:36.167154+0000","last_peered":"2024-04-26T22:28:36.167154+0000","last_clean":"2024-04-26T22:28:36.167154+0000","last_became_active":"2024-04-26T22:28:36.160683+0000","last_became_peered":"2024-04-26T22:28:36.160683+0000","last_unstale":"2024-04-26T22:28:36.167154+0000","last_undegraded":"2024-04-26T22:28:36.167154+0000","last_fullsized":"2024-04-26T22:28:36.167154+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153574+0000","last_change":"2024-04-26T22:28:36.153574+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153574+0000","last_undegraded":"2024-04-26T22:28:36.153574+0000","last_fullsized":"2024-04-26T22:28:36.153574+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164527+0000","last_change":"2024-04-26T22:28:36.162766+0000","last_active":"2024-04-26T22:28:36.164527+0000","last_peered":"2024-04-26T22:28:36.164527+0000","last_clean":"2024-04-26T22:28:36.164527+0000","last_became_active":"2024-04-26T22:28:36.162468+0000","last_became_peered":"2024-04-26T22:28:36.162468+0000","last_unstale":"2024-04-26T22:28:36.164527+0000","last_undegraded":"2024-04-26T22:28:36.164527+0000","last_fullsized":"2024-04-26T22:28:36.164527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.158533+0000","last_change":"2024-04-26T22:28:36.158533+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.158533+0000","last_undegraded":"2024-04-26T22:28:36.158533+0000","last_fullsized":"2024-04-26T22:28:36.158533+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164922+0000","last_change":"2024-04-26T22:28:36.162336+0000","last_active":"2024-04-26T22:28:36.164922+0000","last_peered":"2024-04-26T22:28:36.164922+0000","last_clean":"2024-04-26T22:28:36.164922+0000","last_became_active":"2024-04-26T22:28:36.162067+0000","last_became_peered":"2024-04-26T22:28:36.162067+0000","last_unstale":"2024-04-26T22:28:36.164922+0000","last_undegraded":"2024-04-26T22:28:36.164922+0000","last_fullsized":"2024-04-26T22:28:36.164922+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148121+0000","last_change":"2024-04-26T22:28:36.148121+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148121+0000","last_undegraded":"2024-04-26T22:28:36.148121+0000","last_fullsized":"2024-04-26T22:28:36.148121+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164607+0000","last_change":"2024-04-26T22:28:36.161738+0000","last_active":"2024-04-26T22:28:36.164607+0000","last_peered":"2024-04-26T22:28:36.164607+0000","last_clean":"2024-04-26T22:28:36.164607+0000","last_became_active":"2024-04-26T22:28:36.161489+0000","last_became_peered":"2024-04-26T22:28:36.161489+0000","last_unstale":"2024-04-26T22:28:36.164607+0000","last_undegraded":"2024-04-26T22:28:36.164607+0000","last_fullsized":"2024-04-26T22:28:36.164607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:50:38.921239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.150196+0000","last_change":"2024-04-26T22:28:36.150196+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.150196+0000","last_undegraded":"2024-04-26T22:28:36.150196+0000","last_fullsized":"2024-04-26T22:28:36.150196+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164669+0000","last_change":"2024-04-26T22:28:36.159715+0000","last_active":"2024-04-26T22:28:36.164669+0000","last_peered":"2024-04-26T22:28:36.164669+0000","last_clean":"2024-04-26T22:28:36.164669+0000","last_became_active":"2024-04-26T22:28:36.159437+0000","last_became_peered":"2024-04-26T22:28:36.159437+0000","last_unstale":"2024-04-26T22:28:36.164669+0000","last_undegraded":"2024-04-26T22:28:36.164669+0000","last_fullsized":"2024-04-26T22:28:36.164669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149529+0000","last_change":"2024-04-26T22:28:36.149529+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149529+0000","last_undegraded":"2024-04-26T22:28:36.149529+0000","last_fullsized":"2024-04-26T22:28:36.149529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165270+0000","last_change":"2024-04-26T22:28:36.163369+0000","last_active":"2024-04-26T22:28:36.165270+0000","last_peered":"2024-04-26T22:28:36.165270+0000","last_clean":"2024-04-26T22:28:36.165270+0000","last_became_active":"2024-04-26T22:28:36.163060+0000","last_became_peered":"2024-04-26T22:28:36.163060+0000","last_unstale":"2024-04-26T22:28:36.165270+0000","last_undegraded":"2024-04-26T22:28:36.165270+0000","last_fullsized":"2024-04-26T22:28:36.165270+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:58.196234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151707+0000","last_change":"2024-04-26T22:28:36.151707+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151707+0000","last_undegraded":"2024-04-26T22:28:36.151707+0000","last_fullsized":"2024-04-26T22:28:36.151707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.166646+0000","last_change":"2024-04-26T22:28:36.165081+0000","last_active":"2024-04-26T22:28:36.166646+0000","last_peered":"2024-04-26T22:28:36.166646+0000","last_clean":"2024-04-26T22:28:36.166646+0000","last_became_active":"2024-04-26T22:28:36.164792+0000","last_became_peered":"2024-04-26T22:28:36.164792+0000","last_unstale":"2024-04-26T22:28:36.166646+0000","last_undegraded":"2024-04-26T22:28:36.166646+0000","last_fullsized":"2024-04-26T22:28:36.166646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:54.680123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155483+0000","last_change":"2024-04-26T22:28:36.155483+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155483+0000","last_undegraded":"2024-04-26T22:28:36.155483+0000","last_fullsized":"2024-04-26T22:28:36.155483+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162204+0000","last_change":"2024-04-26T22:28:36.158147+0000","last_active":"2024-04-26T22:28:36.162204+0000","last_peered":"2024-04-26T22:28:36.162204+0000","last_clean":"2024-04-26T22:28:36.162204+0000","last_became_active":"2024-04-26T22:28:36.157890+0000","last_became_peered":"2024-04-26T22:28:36.157890+0000","last_unstale":"2024-04-26T22:28:36.162204+0000","last_undegraded":"2024-04-26T22:28:36.162204+0000","last_fullsized":"2024-04-26T22:28:36.162204+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151703+0000","last_change":"2024-04-26T22:28:36.151703+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151703+0000","last_undegraded":"2024-04-26T22:28:36.151703+0000","last_fullsized":"2024-04-26T22:28:36.151703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.165776+0000","last_change":"2024-04-26T22:28:36.164416+0000","last_active":"2024-04-26T22:28:36.165776+0000","last_peered":"2024-04-26T22:28:36.165776+0000","last_clean":"2024-04-26T22:28:36.165776+0000","last_became_active":"2024-04-26T22:28:36.164074+0000","last_became_peered":"2024-04-26T22:28:36.164074+0000","last_unstale":"2024-04-26T22:28:36.165776+0000","last_undegraded":"2024-04-26T22:28:36.165776+0000","last_fullsized":"2024-04-26T22:28:36.165776+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:44.778533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.148440+0000","last_change":"2024-04-26T22:28:36.148440+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.148440+0000","last_undegraded":"2024-04-26T22:28:36.148440+0000","last_fullsized":"2024-04-26T22:28:36.148440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.168241+0000","last_change":"2024-04-26T22:28:36.167663+0000","last_active":"2024-04-26T22:28:36.168241+0000","last_peered":"2024-04-26T22:28:36.168241+0000","last_clean":"2024-04-26T22:28:36.168241+0000","last_became_active":"2024-04-26T22:28:36.167430+0000","last_became_peered":"2024-04-26T22:28:36.167430+0000","last_unstale":"2024-04-26T22:28:36.168241+0000","last_undegraded":"2024-04-26T22:28:36.168241+0000","last_fullsized":"2024-04-26T22:28:36.168241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:14.255439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146555+0000","last_change":"2024-04-26T22:28:36.146555+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146555+0000","last_undegraded":"2024-04-26T22:28:36.146555+0000","last_fullsized":"2024-04-26T22:28:36.146555+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155478+0000","last_change":"2024-04-26T22:28:36.152997+0000","last_active":"2024-04-26T22:28:36.155478+0000","last_peered":"2024-04-26T22:28:36.155478+0000","last_clean":"2024-04-26T22:28:36.155478+0000","last_became_active":"2024-04-26T22:28:36.152694+0000","last_became_peered":"2024-04-26T22:28:36.152694+0000","last_unstale":"2024-04-26T22:28:36.155478+0000","last_undegraded":"2024-04-26T22:28:36.155478+0000","last_fullsized":"2024-04-26T22:28:36.155478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:46:42.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153502+0000","last_change":"2024-04-26T22:28:36.153502+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153502+0000","last_undegraded":"2024-04-26T22:28:36.153502+0000","last_fullsized":"2024-04-26T22:28:36.153502+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164982+0000","last_change":"2024-04-26T22:28:36.162940+0000","last_active":"2024-04-26T22:28:36.164982+0000","last_peered":"2024-04-26T22:28:36.164982+0000","last_clean":"2024-04-26T22:28:36.164982+0000","last_became_active":"2024-04-26T22:28:36.162676+0000","last_became_peered":"2024-04-26T22:28:36.162676+0000","last_unstale":"2024-04-26T22:28:36.164982+0000","last_undegraded":"2024-04-26T22:28:36.164982+0000","last_fullsized":"2024-04-26T22:28:36.164982+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153988+0000","last_change":"2024-04-26T22:28:36.153988+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153988+0000","last_undegraded":"2024-04-26T22:28:36.153988+0000","last_fullsized":"2024-04-26T22:28:36.153988+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167075+0000","last_change":"2024-04-26T22:28:36.157167+0000","last_active":"2024-04-26T22:28:36.167075+0000","last_peered":"2024-04-26T22:28:36.167075+0000","last_clean":"2024-04-26T22:28:36.167075+0000","last_became_active":"2024-04-26T22:28:36.156867+0000","last_became_peered":"2024-04-26T22:28:36.156867+0000","last_unstale":"2024-04-26T22:28:36.167075+0000","last_undegraded":"2024-04-26T22:28:36.167075+0000","last_fullsized":"2024-04-26T22:28:36.167075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.154215+0000","last_change":"2024-04-26T22:28:36.154215+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.154215+0000","last_undegraded":"2024-04-26T22:28:36.154215+0000","last_fullsized":"2024-04-26T22:28:36.154215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.163612+0000","last_change":"2024-04-26T22:28:36.160489+0000","last_active":"2024-04-26T22:28:36.163612+0000","last_peered":"2024-04-26T22:28:36.163612+0000","last_clean":"2024-04-26T22:28:36.163612+0000","last_became_active":"2024-04-26T22:28:36.160227+0000","last_became_peered":"2024-04-26T22:28:36.160227+0000","last_unstale":"2024-04-26T22:28:36.163612+0000","last_undegraded":"2024-04-26T22:28:36.163612+0000","last_fullsized":"2024-04-26T22:28:36.163612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:23.581702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149391+0000","last_change":"2024-04-26T22:28:36.149391+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149391+0000","last_undegraded":"2024-04-26T22:28:36.149391+0000","last_fullsized":"2024-04-26T22:28:36.149391+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.159929+0000","last_change":"2024-04-26T22:28:36.156431+0000","last_active":"2024-04-26T22:28:36.159929+0000","last_peered":"2024-04-26T22:28:36.159929+0000","last_clean":"2024-04-26T22:28:36.159929+0000","last_became_active":"2024-04-26T22:28:36.156129+0000","last_became_peered":"2024-04-26T22:28:36.156129+0000","last_unstale":"2024-04-26T22:28:36.159929+0000","last_undegraded":"2024-04-26T22:28:36.159929+0000","last_fullsized":"2024-04-26T22:28:36.159929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:07.246183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146638+0000","last_change":"2024-04-26T22:28:36.146638+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146638+0000","last_undegraded":"2024-04-26T22:28:36.146638+0000","last_fullsized":"2024-04-26T22:28:36.146638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167053+0000","last_change":"2024-04-26T22:28:36.156716+0000","last_active":"2024-04-26T22:28:36.167053+0000","last_peered":"2024-04-26T22:28:36.167053+0000","last_clean":"2024-04-26T22:28:36.167053+0000","last_became_active":"2024-04-26T22:28:36.156424+0000","last_became_peered":"2024-04-26T22:28:36.156424+0000","last_unstale":"2024-04-26T22:28:36.167053+0000","last_undegraded":"2024-04-26T22:28:36.167053+0000","last_fullsized":"2024-04-26T22:28:36.167053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146530+0000","last_change":"2024-04-26T22:28:36.146530+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146530+0000","last_undegraded":"2024-04-26T22:28:36.146530+0000","last_fullsized":"2024-04-26T22:28:36.146530+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.160043+0000","last_change":"2024-04-26T22:28:36.157532+0000","last_active":"2024-04-26T22:28:36.160043+0000","last_peered":"2024-04-26T22:28:36.160043+0000","last_clean":"2024-04-26T22:28:36.160043+0000","last_became_active":"2024-04-26T22:28:36.157238+0000","last_became_peered":"2024-04-26T22:28:36.157238+0000","last_unstale":"2024-04-26T22:28:36.160043+0000","last_undegraded":"2024-04-26T22:28:36.160043+0000","last_fullsized":"2024-04-26T22:28:36.160043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:16.612120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.151105+0000","last_change":"2024-04-26T22:28:36.151105+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.151105+0000","last_undegraded":"2024-04-26T22:28:36.151105+0000","last_fullsized":"2024-04-26T22:28:36.151105+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.162295+0000","last_change":"2024-04-26T22:28:36.157860+0000","last_active":"2024-04-26T22:28:36.162295+0000","last_peered":"2024-04-26T22:28:36.162295+0000","last_clean":"2024-04-26T22:28:36.162295+0000","last_became_active":"2024-04-26T22:28:36.157588+0000","last_became_peered":"2024-04-26T22:28:36.157588+0000","last_unstale":"2024-04-26T22:28:36.162295+0000","last_undegraded":"2024-04-26T22:28:36.162295+0000","last_fullsized":"2024-04-26T22:28:36.162295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153545+0000","last_change":"2024-04-26T22:28:36.153545+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153545+0000","last_undegraded":"2024-04-26T22:28:36.153545+0000","last_fullsized":"2024-04-26T22:28:36.153545+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167355+0000","last_change":"2024-04-26T22:28:36.165811+0000","last_active":"2024-04-26T22:28:36.167355+0000","last_peered":"2024-04-26T22:28:36.167355+0000","last_clean":"2024-04-26T22:28:36.167355+0000","last_became_active":"2024-04-26T22:28:36.165461+0000","last_became_peered":"2024-04-26T22:28:36.165461+0000","last_unstale":"2024-04-26T22:28:36.167355+0000","last_undegraded":"2024-04-26T22:28:36.167355+0000","last_fullsized":"2024-04-26T22:28:36.167355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:18.602353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.153458+0000","last_change":"2024-04-26T22:28:36.153458+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.153458+0000","last_undegraded":"2024-04-26T22:28:36.153458+0000","last_fullsized":"2024-04-26T22:28:36.153458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164692+0000","last_change":"2024-04-26T22:28:36.159775+0000","last_active":"2024-04-26T22:28:36.164692+0000","last_peered":"2024-04-26T22:28:36.164692+0000","last_clean":"2024-04-26T22:28:36.164692+0000","last_became_active":"2024-04-26T22:28:36.159490+0000","last_became_peered":"2024-04-26T22:28:36.159490+0000","last_unstale":"2024-04-26T22:28:36.164692+0000","last_undegraded":"2024-04-26T22:28:36.164692+0000","last_fullsized":"2024-04-26T22:28:36.164692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.149079+0000","last_change":"2024-04-26T22:28:36.149079+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.149079+0000","last_undegraded":"2024-04-26T22:28:36.149079+0000","last_fullsized":"2024-04-26T22:28:36.149079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.155941+0000","last_change":"2024-04-26T22:28:36.154743+0000","last_active":"2024-04-26T22:28:36.155941+0000","last_peered":"2024-04-26T22:28:36.155941+0000","last_clean":"2024-04-26T22:28:36.155941+0000","last_became_active":"2024-04-26T22:28:36.154433+0000","last_became_peered":"2024-04-26T22:28:36.154433+0000","last_unstale":"2024-04-26T22:28:36.155941+0000","last_undegraded":"2024-04-26T22:28:36.155941+0000","last_fullsized":"2024-04-26T22:28:36.155941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146336+0000","last_change":"2024-04-26T22:28:36.146336+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146336+0000","last_undegraded":"2024-04-26T22:28:36.146336+0000","last_fullsized":"2024-04-26T22:28:36.146336+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164732+0000","last_change":"2024-04-26T22:28:36.160465+0000","last_active":"2024-04-26T22:28:36.164732+0000","last_peered":"2024-04-26T22:28:36.164732+0000","last_clean":"2024-04-26T22:28:36.164732+0000","last_became_active":"2024-04-26T22:28:36.160206+0000","last_became_peered":"2024-04-26T22:28:36.160206+0000","last_unstale":"2024-04-26T22:28:36.164732+0000","last_undegraded":"2024-04-26T22:28:36.164732+0000","last_fullsized":"2024-04-26T22:28:36.164732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:47.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.146383+0000","last_change":"2024-04-26T22:28:36.146383+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.146383+0000","last_undegraded":"2024-04-26T22:28:36.146383+0000","last_fullsized":"2024-04-26T22:28:36.146383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164966+0000","last_change":"2024-04-26T22:28:36.163006+0000","last_active":"2024-04-26T22:28:36.164966+0000","last_peered":"2024-04-26T22:28:36.164966+0000","last_clean":"2024-04-26T22:28:36.164966+0000","last_became_active":"2024-04-26T22:28:36.162735+0000","last_became_peered":"2024-04-26T22:28:36.162735+0000","last_unstale":"2024-04-26T22:28:36.164966+0000","last_undegraded":"2024-04-26T22:28:36.164966+0000","last_fullsized":"2024-04-26T22:28:36.164966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:44.870984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.147713+0000","last_change":"2024-04-26T22:28:36.147713+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.147713+0000","last_undegraded":"2024-04-26T22:28:36.147713+0000","last_fullsized":"2024-04-26T22:28:36.147713+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.161202+0000","last_change":"2024-04-26T22:28:36.159382+0000","last_active":"2024-04-26T22:28:36.161202+0000","last_peered":"2024-04-26T22:28:36.161202+0000","last_clean":"2024-04-26T22:28:36.161202+0000","last_became_active":"2024-04-26T22:28:36.159016+0000","last_became_peered":"2024-04-26T22:28:36.159016+0000","last_unstale":"2024-04-26T22:28:36.161202+0000","last_undegraded":"2024-04-26T22:28:36.161202+0000","last_fullsized":"2024-04-26T22:28:36.161202+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.156626+0000","last_change":"2024-04-26T22:28:36.156626+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.156626+0000","last_undegraded":"2024-04-26T22:28:36.156626+0000","last_fullsized":"2024-04-26T22:28:36.156626+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.164507+0000","last_change":"2024-04-26T22:28:36.162470+0000","last_active":"2024-04-26T22:28:36.164507+0000","last_peered":"2024-04-26T22:28:36.164507+0000","last_clean":"2024-04-26T22:28:36.164507+0000","last_became_active":"2024-04-26T22:28:36.162164+0000","last_became_peered":"2024-04-26T22:28:36.162164+0000","last_unstale":"2024-04-26T22:28:36.164507+0000","last_undegraded":"2024-04-26T22:28:36.164507+0000","last_fullsized":"2024-04-26T22:28:36.164507+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:42.027471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-04-26T22:28:36.155799+0000","last_change":"2024-04-26T22:28:36.155799+0000","last_active":"2024-04-26T22:28:36.133205+0000","last_peered":"2024-04-26T22:28:36.133205+0000","last_clean":"2024-04-26T22:28:36.133205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T22:28:36.155799+0000","last_undegraded":"2024-04-26T22:28:36.155799+0000","last_fullsized":"2024-04-26T22:28:36.155799+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.167245+0000","last_change":"2024-04-26T22:28:36.162279+0000","last_active":"2024-04-26T22:28:36.167245+0000","last_peered":"2024-04-26T22:28:36.167245+0000","last_clean":"2024-04-26T22:28:36.167245+0000","last_became_active":"2024-04-26T22:28:36.161730+0000","last_became_peered":"2024-04-26T22:28:36.161730+0000","last_unstale":"2024-04-26T22:28:36.167245+0000","last_undegraded":"2024-04-26T22:28:36.167245+0000","last_fullsized":"2024-04-26T22:28:36.167245+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:32.082046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-26T22:28:36.152298+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:36.152298+0000","last_peered":"2024-04-26T22:28:36.152298+0000","last_clean":"2024-04-26T22:28:36.152298+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:36.152298+0000","last_undegraded":"2024-04-26T22:28:36.152298+0000","last_fullsized":"2024-04-26T22:28:36.152298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":774224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":466944,"data_stored":183856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":190123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:42.796 INFO:tasks.ceph.mds.b.smithi174.stderr:2024-04-26T22:28:42.795+0000 7f35b58f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T22:28:43.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:28:43.880 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:28:44.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T22:28:44.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.723+0000 7fea6e409640 1 Processor -- start 2024-04-26T22:28:44.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.723+0000 7fea6e409640 1 -- start start 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea6806c240 0x7fea6806fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 0x7fea6806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 0x7fea680ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fea6806cb60 con 0x7fea6806c240 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fea680b59b0 con 0x7fea680b1480 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6e409640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fea68070350 con 0x7fea6815f4f0 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea67fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea6806c240 0x7fea6806fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea67fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fea6806c240 0x7fea6806fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38120/0 (socket says 172.21.15.62:38120) 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea67fff640 1 -- 172.21.15.62:0/4027838803 learned_addr learned my addr 172.21.15.62:0/4027838803 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea6c9a6640 1 --2- 172.21.15.62:0/4027838803 >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 0x7fea680ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.724+0000 7fea677fe640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 0x7fea6806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 -- 172.21.15.62:0/4027838803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 msgr2=0x7fea6806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 0x7fea6806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 -- 172.21.15.62:0/4027838803 >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 msgr2=0x7fea680ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 --2- 172.21.15.62:0/4027838803 >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 0x7fea680ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 -- 172.21.15.62:0/4027838803 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea680b5c70 con 0x7fea6806c240 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea677fe640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 0x7fea6806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:44.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea6c9a6640 1 --2- 172.21.15.62:0/4027838803 >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 0x7fea680ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:44.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.725+0000 7fea67fff640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806c240 0x7fea6806fd70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea68071d30 tx=0x7fea5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c70475b6f7c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:44.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea66ffd640 1 -- 172.21.15.62:0/4027838803 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50037070 con 0x7fea6806c240 2024-04-26T22:28:44.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea66ffd640 1 -- 172.21.15.62:0/4027838803 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea50032440 con 0x7fea6806c240 2024-04-26T22:28:44.727 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea66ffd640 1 -- 172.21.15.62:0/4027838803 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50032810 con 0x7fea6806c240 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 -- 172.21.15.62:0/4027838803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806c240 msgr2=0x7fea6806fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806c240 0x7fea6806fd70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea68071d30 tx=0x7fea5002a1e0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 -- 172.21.15.62:0/4027838803 shutdown_connections 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 --2- 172.21.15.62:0/4027838803 >> v2:172.21.15.174:3300/0 conn(0x7fea680b1480 0x7fea680ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6815f4f0 0x7fea6806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 --2- 172.21.15.62:0/4027838803 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea6806c240 0x7fea6806fd70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 -- 172.21.15.62:0/4027838803 >> 172.21.15.62:0/4027838803 conn(0x7fea680b3b70 msgr2=0x7fea680b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 -- 172.21.15.62:0/4027838803 shutdown_connections 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.726+0000 7fea6e409640 1 -- 172.21.15.62:0/4027838803 wait complete. 2024-04-26T22:28:44.728 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.727+0000 7fea6e409640 1 Processor -- start 2024-04-26T22:28:44.731 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 -- start start 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6806c240 0x7fea68157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 0x7fea68156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fea680aee00 con 0x7fea680b1480 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea680b6230 con 0x7fea6815f4f0 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.728+0000 7fea6e409640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fea680b0fc0 con 0x7fea6806c240 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38122/0 (socket says 172.21.15.62:38122) 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 -- 172.21.15.62:0/383998906 learned_addr learned my addr 172.21.15.62:0/383998906 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea67fff640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6806c240 0x7fea68157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.733 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea6c9a6640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 0x7fea68156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 -- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6806c240 msgr2=0x7fea68157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6806c240 0x7fea68157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 -- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 msgr2=0x7fea68156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 0x7fea68156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 -- 172.21.15.62:0/383998906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea68156e30 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea6c9a6640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 0x7fea68156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea677fe640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d7d0 tx=0x7fea5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58004030 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea580041d0 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea68157110 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5800b6c0 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.729+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681576d0 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.730+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6806a360 con 0x7fea680b1480 2024-04-26T22:28:44.734 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.731+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fea5800b860 con 0x7fea680b1480 2024-04-26T22:28:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.731+0000 7fea64ff9640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 0x7fea4806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.732+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fea5808bd90 con 0x7fea680b1480 2024-04-26T22:28:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.732+0000 7fea67fff640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 0x7fea4806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:44.735 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.732+0000 7fea67fff640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 0x7fea4806ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea50034000 tx=0x7fea5002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:44.736 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.736+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea5805a1d0 con 0x7fea680b1480 2024-04-26T22:28:44.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.917+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea68157ad0 con 0x7fea4806c980 2024-04-26T22:28:44.923 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.923+0000 7fea64ff9640 1 -- 172.21.15.62:0/383998906 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fea68157ad0 con 0x7fea4806c980 2024-04-26T22:28:44.926 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:44.927 INFO:teuthology.orchestra.run.smithi062.stderr:dumped all 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 msgr2=0x7fea4806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 0x7fea4806ee40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea50034000 tx=0x7fea5002e000 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 msgr2=0x7fea68161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea5800d7d0 tx=0x7fea5800dc90 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 shutdown_connections 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fea4806c980 0x7fea4806ee40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea6815f4f0 0x7fea68156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea680b1480 0x7fea68161c70 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 --2- 172.21.15.62:0/383998906 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea6806c240 0x7fea68157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 >> 172.21.15.62:0/383998906 conn(0x7fea680b3b70 msgr2=0x7fea680b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 shutdown_connections 2024-04-26T22:28:44.929 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:44.927+0000 7fea6e409640 1 -- 172.21.15.62:0/383998906 wait complete. 2024-04-26T22:28:44.938 INFO:teuthology.orchestra.run.smithi062.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-04-26T22:28:43.659487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221320,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753400,"statfs":{"total":773094113280,"available":772867481600,"internally_reserved":0,"allocated":5283840,"data_stored":2811140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.512960"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148059+0000","last_change":"2024-04-26T22:28:37.167640+0000","last_active":"2024-04-26T22:28:38.148059+0000","last_peered":"2024-04-26T22:28:38.148059+0000","last_clean":"2024-04-26T22:28:38.148059+0000","last_became_active":"2024-04-26T22:28:37.167296+0000","last_became_peered":"2024-04-26T22:28:37.167296+0000","last_unstale":"2024-04-26T22:28:38.148059+0000","last_undegraded":"2024-04-26T22:28:38.148059+0000","last_fullsized":"2024-04-26T22:28:38.148059+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:58.429758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148299+0000","last_change":"2024-04-26T22:28:37.168309+0000","last_active":"2024-04-26T22:28:38.148299+0000","last_peered":"2024-04-26T22:28:38.148299+0000","last_clean":"2024-04-26T22:28:38.148299+0000","last_became_active":"2024-04-26T22:28:37.168016+0000","last_became_peered":"2024-04-26T22:28:37.168016+0000","last_unstale":"2024-04-26T22:28:38.148299+0000","last_undegraded":"2024-04-26T22:28:38.148299+0000","last_fullsized":"2024-04-26T22:28:38.148299+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:43.310145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148464+0000","last_change":"2024-04-26T22:28:37.165336+0000","last_active":"2024-04-26T22:28:38.148464+0000","last_peered":"2024-04-26T22:28:38.148464+0000","last_clean":"2024-04-26T22:28:38.148464+0000","last_became_active":"2024-04-26T22:28:37.165000+0000","last_became_peered":"2024-04-26T22:28:37.165000+0000","last_unstale":"2024-04-26T22:28:38.148464+0000","last_undegraded":"2024-04-26T22:28:38.148464+0000","last_fullsized":"2024-04-26T22:28:38.148464+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:50.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320068+0000","last_change":"2024-04-26T22:28:37.169503+0000","last_active":"2024-04-26T22:28:38.320068+0000","last_peered":"2024-04-26T22:28:38.320068+0000","last_clean":"2024-04-26T22:28:38.320068+0000","last_became_active":"2024-04-26T22:28:37.169218+0000","last_became_peered":"2024-04-26T22:28:37.169218+0000","last_unstale":"2024-04-26T22:28:38.320068+0000","last_undegraded":"2024-04-26T22:28:38.320068+0000","last_fullsized":"2024-04-26T22:28:38.320068+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:32.284968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148640+0000","last_change":"2024-04-26T22:28:37.163007+0000","last_active":"2024-04-26T22:28:38.148640+0000","last_peered":"2024-04-26T22:28:38.148640+0000","last_clean":"2024-04-26T22:28:38.148640+0000","last_became_active":"2024-04-26T22:28:37.162759+0000","last_became_peered":"2024-04-26T22:28:37.162759+0000","last_unstale":"2024-04-26T22:28:38.148640+0000","last_undegraded":"2024-04-26T22:28:38.148640+0000","last_fullsized":"2024-04-26T22:28:38.148640+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.197577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319704+0000","last_change":"2024-04-26T22:28:37.170624+0000","last_active":"2024-04-26T22:28:38.319704+0000","last_peered":"2024-04-26T22:28:38.319704+0000","last_clean":"2024-04-26T22:28:38.319704+0000","last_became_active":"2024-04-26T22:28:37.170346+0000","last_became_peered":"2024-04-26T22:28:37.170346+0000","last_unstale":"2024-04-26T22:28:38.319704+0000","last_undegraded":"2024-04-26T22:28:38.319704+0000","last_fullsized":"2024-04-26T22:28:38.319704+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:20.177510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147489+0000","last_change":"2024-04-26T22:28:37.168730+0000","last_active":"2024-04-26T22:28:38.147489+0000","last_peered":"2024-04-26T22:28:38.147489+0000","last_clean":"2024-04-26T22:28:38.147489+0000","last_became_active":"2024-04-26T22:28:37.168438+0000","last_became_peered":"2024-04-26T22:28:37.168438+0000","last_unstale":"2024-04-26T22:28:38.147489+0000","last_undegraded":"2024-04-26T22:28:38.147489+0000","last_fullsized":"2024-04-26T22:28:38.147489+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:16.948027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319059+0000","last_change":"2024-04-26T22:28:37.171484+0000","last_active":"2024-04-26T22:28:38.319059+0000","last_peered":"2024-04-26T22:28:38.319059+0000","last_clean":"2024-04-26T22:28:38.319059+0000","last_became_active":"2024-04-26T22:28:37.170563+0000","last_became_peered":"2024-04-26T22:28:37.170563+0000","last_unstale":"2024-04-26T22:28:38.319059+0000","last_undegraded":"2024-04-26T22:28:38.319059+0000","last_fullsized":"2024-04-26T22:28:38.319059+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:42.126368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320429+0000","last_change":"2024-04-26T22:28:37.165663+0000","last_active":"2024-04-26T22:28:38.320429+0000","last_peered":"2024-04-26T22:28:38.320429+0000","last_clean":"2024-04-26T22:28:38.320429+0000","last_became_active":"2024-04-26T22:28:37.164803+0000","last_became_peered":"2024-04-26T22:28:37.164803+0000","last_unstale":"2024-04-26T22:28:38.320429+0000","last_undegraded":"2024-04-26T22:28:38.320429+0000","last_fullsized":"2024-04-26T22:28:38.320429+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:02.822292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320665+0000","last_change":"2024-04-26T22:28:37.172016+0000","last_active":"2024-04-26T22:28:38.320665+0000","last_peered":"2024-04-26T22:28:38.320665+0000","last_clean":"2024-04-26T22:28:38.320665+0000","last_became_active":"2024-04-26T22:28:37.171431+0000","last_became_peered":"2024-04-26T22:28:37.171431+0000","last_unstale":"2024-04-26T22:28:38.320665+0000","last_undegraded":"2024-04-26T22:28:38.320665+0000","last_fullsized":"2024-04-26T22:28:38.320665+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:26:08.170352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320385+0000","last_change":"2024-04-26T22:28:37.163410+0000","last_active":"2024-04-26T22:28:38.320385+0000","last_peered":"2024-04-26T22:28:38.320385+0000","last_clean":"2024-04-26T22:28:38.320385+0000","last_became_active":"2024-04-26T22:28:37.163107+0000","last_became_peered":"2024-04-26T22:28:37.163107+0000","last_unstale":"2024-04-26T22:28:38.320385+0000","last_undegraded":"2024-04-26T22:28:38.320385+0000","last_fullsized":"2024-04-26T22:28:38.320385+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:41.641104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148754+0000","last_change":"2024-04-26T22:28:37.163366+0000","last_active":"2024-04-26T22:28:38.148754+0000","last_peered":"2024-04-26T22:28:38.148754+0000","last_clean":"2024-04-26T22:28:38.148754+0000","last_became_active":"2024-04-26T22:28:37.163106+0000","last_became_peered":"2024-04-26T22:28:37.163106+0000","last_unstale":"2024-04-26T22:28:38.148754+0000","last_undegraded":"2024-04-26T22:28:38.148754+0000","last_fullsized":"2024-04-26T22:28:38.148754+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:42.126368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145457+0000","last_change":"2024-04-26T22:28:37.160461+0000","last_active":"2024-04-26T22:28:38.145457+0000","last_peered":"2024-04-26T22:28:38.145457+0000","last_clean":"2024-04-26T22:28:38.145457+0000","last_became_active":"2024-04-26T22:28:37.160182+0000","last_became_peered":"2024-04-26T22:28:37.160182+0000","last_unstale":"2024-04-26T22:28:38.145457+0000","last_undegraded":"2024-04-26T22:28:38.145457+0000","last_fullsized":"2024-04-26T22:28:38.145457+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:48.436512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320841+0000","last_change":"2024-04-26T22:28:37.164345+0000","last_active":"2024-04-26T22:28:38.320841+0000","last_peered":"2024-04-26T22:28:38.320841+0000","last_clean":"2024-04-26T22:28:38.320841+0000","last_became_active":"2024-04-26T22:28:37.164074+0000","last_became_peered":"2024-04-26T22:28:37.164074+0000","last_unstale":"2024-04-26T22:28:38.320841+0000","last_undegraded":"2024-04-26T22:28:38.320841+0000","last_fullsized":"2024-04-26T22:28:38.320841+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:14.916211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319805+0000","last_change":"2024-04-26T22:28:37.170504+0000","last_active":"2024-04-26T22:28:38.319805+0000","last_peered":"2024-04-26T22:28:38.319805+0000","last_clean":"2024-04-26T22:28:38.319805+0000","last_became_active":"2024-04-26T22:28:37.170193+0000","last_became_peered":"2024-04-26T22:28:37.170193+0000","last_unstale":"2024-04-26T22:28:38.319805+0000","last_undegraded":"2024-04-26T22:28:38.319805+0000","last_fullsized":"2024-04-26T22:28:38.319805+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:00.877975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319336+0000","last_change":"2024-04-26T22:28:37.164339+0000","last_active":"2024-04-26T22:28:38.319336+0000","last_peered":"2024-04-26T22:28:38.319336+0000","last_clean":"2024-04-26T22:28:38.319336+0000","last_became_active":"2024-04-26T22:28:37.164061+0000","last_became_peered":"2024-04-26T22:28:37.164061+0000","last_unstale":"2024-04-26T22:28:38.319336+0000","last_undegraded":"2024-04-26T22:28:38.319336+0000","last_fullsized":"2024-04-26T22:28:38.319336+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:48.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148776+0000","last_change":"2024-04-26T22:28:37.163354+0000","last_active":"2024-04-26T22:28:38.148776+0000","last_peered":"2024-04-26T22:28:38.148776+0000","last_clean":"2024-04-26T22:28:38.148776+0000","last_became_active":"2024-04-26T22:28:37.161756+0000","last_became_peered":"2024-04-26T22:28:37.161756+0000","last_unstale":"2024-04-26T22:28:38.148776+0000","last_undegraded":"2024-04-26T22:28:38.148776+0000","last_fullsized":"2024-04-26T22:28:38.148776+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:49.178405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148526+0000","last_change":"2024-04-26T22:28:37.167348+0000","last_active":"2024-04-26T22:28:38.148526+0000","last_peered":"2024-04-26T22:28:38.148526+0000","last_clean":"2024-04-26T22:28:38.148526+0000","last_became_active":"2024-04-26T22:28:37.167076+0000","last_became_peered":"2024-04-26T22:28:37.167076+0000","last_unstale":"2024-04-26T22:28:38.148526+0000","last_undegraded":"2024-04-26T22:28:38.148526+0000","last_fullsized":"2024-04-26T22:28:38.148526+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:43.310145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319797+0000","last_change":"2024-04-26T22:28:37.163045+0000","last_active":"2024-04-26T22:28:38.319797+0000","last_peered":"2024-04-26T22:28:38.319797+0000","last_clean":"2024-04-26T22:28:38.319797+0000","last_became_active":"2024-04-26T22:28:37.162794+0000","last_became_peered":"2024-04-26T22:28:37.162794+0000","last_unstale":"2024-04-26T22:28:38.319797+0000","last_undegraded":"2024-04-26T22:28:38.319797+0000","last_fullsized":"2024-04-26T22:28:38.319797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:55.682939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145638+0000","last_change":"2024-04-26T22:28:37.162156+0000","last_active":"2024-04-26T22:28:38.145638+0000","last_peered":"2024-04-26T22:28:38.145638+0000","last_clean":"2024-04-26T22:28:38.145638+0000","last_became_active":"2024-04-26T22:28:37.161865+0000","last_became_peered":"2024-04-26T22:28:37.161865+0000","last_unstale":"2024-04-26T22:28:38.145638+0000","last_undegraded":"2024-04-26T22:28:38.145638+0000","last_fullsized":"2024-04-26T22:28:38.145638+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:10.954865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319662+0000","last_change":"2024-04-26T22:28:37.157849+0000","last_active":"2024-04-26T22:28:38.319662+0000","last_peered":"2024-04-26T22:28:38.319662+0000","last_clean":"2024-04-26T22:28:38.319662+0000","last_became_active":"2024-04-26T22:28:37.157541+0000","last_became_peered":"2024-04-26T22:28:37.157541+0000","last_unstale":"2024-04-26T22:28:38.319662+0000","last_undegraded":"2024-04-26T22:28:38.319662+0000","last_fullsized":"2024-04-26T22:28:38.319662+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:59.951982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320832+0000","last_change":"2024-04-26T22:28:37.172550+0000","last_active":"2024-04-26T22:28:38.320832+0000","last_peered":"2024-04-26T22:28:38.320832+0000","last_clean":"2024-04-26T22:28:38.320832+0000","last_became_active":"2024-04-26T22:28:37.172207+0000","last_became_peered":"2024-04-26T22:28:37.172207+0000","last_unstale":"2024-04-26T22:28:38.320832+0000","last_undegraded":"2024-04-26T22:28:38.320832+0000","last_fullsized":"2024-04-26T22:28:38.320832+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:19.615007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319349+0000","last_change":"2024-04-26T22:28:37.170000+0000","last_active":"2024-04-26T22:28:38.319349+0000","last_peered":"2024-04-26T22:28:38.319349+0000","last_clean":"2024-04-26T22:28:38.319349+0000","last_became_active":"2024-04-26T22:28:37.169745+0000","last_became_peered":"2024-04-26T22:28:37.169745+0000","last_unstale":"2024-04-26T22:28:38.319349+0000","last_undegraded":"2024-04-26T22:28:38.319349+0000","last_fullsized":"2024-04-26T22:28:38.319349+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:37.491003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319333+0000","last_change":"2024-04-26T22:28:37.170170+0000","last_active":"2024-04-26T22:28:38.319333+0000","last_peered":"2024-04-26T22:28:38.319333+0000","last_clean":"2024-04-26T22:28:38.319333+0000","last_became_active":"2024-04-26T22:28:37.169915+0000","last_became_peered":"2024-04-26T22:28:37.169915+0000","last_unstale":"2024-04-26T22:28:38.319333+0000","last_undegraded":"2024-04-26T22:28:38.319333+0000","last_fullsized":"2024-04-26T22:28:38.319333+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:36:33.084862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319807+0000","last_change":"2024-04-26T22:28:37.166210+0000","last_active":"2024-04-26T22:28:38.319807+0000","last_peered":"2024-04-26T22:28:38.319807+0000","last_clean":"2024-04-26T22:28:38.319807+0000","last_became_active":"2024-04-26T22:28:37.165951+0000","last_became_peered":"2024-04-26T22:28:37.165951+0000","last_unstale":"2024-04-26T22:28:38.319807+0000","last_undegraded":"2024-04-26T22:28:38.319807+0000","last_fullsized":"2024-04-26T22:28:38.319807+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:58.894271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145605+0000","last_change":"2024-04-26T22:28:37.159653+0000","last_active":"2024-04-26T22:28:38.145605+0000","last_peered":"2024-04-26T22:28:38.145605+0000","last_clean":"2024-04-26T22:28:38.145605+0000","last_became_active":"2024-04-26T22:28:37.159384+0000","last_became_peered":"2024-04-26T22:28:37.159384+0000","last_unstale":"2024-04-26T22:28:38.145605+0000","last_undegraded":"2024-04-26T22:28:38.145605+0000","last_fullsized":"2024-04-26T22:28:38.145605+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:57.220616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148652+0000","last_change":"2024-04-26T22:28:37.169020+0000","last_active":"2024-04-26T22:28:38.148652+0000","last_peered":"2024-04-26T22:28:38.148652+0000","last_clean":"2024-04-26T22:28:38.148652+0000","last_became_active":"2024-04-26T22:28:37.168731+0000","last_became_peered":"2024-04-26T22:28:37.168731+0000","last_unstale":"2024-04-26T22:28:38.148652+0000","last_undegraded":"2024-04-26T22:28:38.148652+0000","last_fullsized":"2024-04-26T22:28:38.148652+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:11.197577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148907+0000","last_change":"2024-04-26T22:28:37.161894+0000","last_active":"2024-04-26T22:28:38.148907+0000","last_peered":"2024-04-26T22:28:38.148907+0000","last_clean":"2024-04-26T22:28:38.148907+0000","last_became_active":"2024-04-26T22:28:37.161592+0000","last_became_peered":"2024-04-26T22:28:37.161592+0000","last_unstale":"2024-04-26T22:28:38.148907+0000","last_undegraded":"2024-04-26T22:28:38.148907+0000","last_fullsized":"2024-04-26T22:28:38.148907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:49.178405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319643+0000","last_change":"2024-04-26T22:28:37.164376+0000","last_active":"2024-04-26T22:28:38.319643+0000","last_peered":"2024-04-26T22:28:38.319643+0000","last_clean":"2024-04-26T22:28:38.319643+0000","last_became_active":"2024-04-26T22:28:37.163861+0000","last_became_peered":"2024-04-26T22:28:37.163861+0000","last_unstale":"2024-04-26T22:28:38.319643+0000","last_undegraded":"2024-04-26T22:28:38.319643+0000","last_fullsized":"2024-04-26T22:28:38.319643+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:24.584518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147870+0000","last_change":"2024-04-26T22:28:37.167490+0000","last_active":"2024-04-26T22:28:38.147870+0000","last_peered":"2024-04-26T22:28:38.147870+0000","last_clean":"2024-04-26T22:28:38.147870+0000","last_became_active":"2024-04-26T22:28:37.167192+0000","last_became_peered":"2024-04-26T22:28:37.167192+0000","last_unstale":"2024-04-26T22:28:38.147870+0000","last_undegraded":"2024-04-26T22:28:38.147870+0000","last_fullsized":"2024-04-26T22:28:38.147870+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:55.682939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319948+0000","last_change":"2024-04-26T22:28:37.166532+0000","last_active":"2024-04-26T22:28:38.319948+0000","last_peered":"2024-04-26T22:28:38.319948+0000","last_clean":"2024-04-26T22:28:38.319948+0000","last_became_active":"2024-04-26T22:28:37.166253+0000","last_became_peered":"2024-04-26T22:28:37.166253+0000","last_unstale":"2024-04-26T22:28:38.319948+0000","last_undegraded":"2024-04-26T22:28:38.319948+0000","last_fullsized":"2024-04-26T22:28:38.319948+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:22.171736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145384+0000","last_change":"2024-04-26T22:28:37.156543+0000","last_active":"2024-04-26T22:28:38.145384+0000","last_peered":"2024-04-26T22:28:38.145384+0000","last_clean":"2024-04-26T22:28:38.145384+0000","last_became_active":"2024-04-26T22:28:37.156273+0000","last_became_peered":"2024-04-26T22:28:37.156273+0000","last_unstale":"2024-04-26T22:28:38.145384+0000","last_undegraded":"2024-04-26T22:28:38.145384+0000","last_fullsized":"2024-04-26T22:28:38.145384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:23:59.838162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319513+0000","last_change":"2024-04-26T22:28:37.153937+0000","last_active":"2024-04-26T22:28:38.319513+0000","last_peered":"2024-04-26T22:28:38.319513+0000","last_clean":"2024-04-26T22:28:38.319513+0000","last_became_active":"2024-04-26T22:28:37.153611+0000","last_became_peered":"2024-04-26T22:28:37.153611+0000","last_unstale":"2024-04-26T22:28:38.319513+0000","last_undegraded":"2024-04-26T22:28:38.319513+0000","last_fullsized":"2024-04-26T22:28:38.319513+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:24.584518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320365+0000","last_change":"2024-04-26T22:28:36.157128+0000","last_active":"2024-04-26T22:28:38.320365+0000","last_peered":"2024-04-26T22:28:38.320365+0000","last_clean":"2024-04-26T22:28:38.320365+0000","last_became_active":"2024-04-26T22:28:36.156854+0000","last_became_peered":"2024-04-26T22:28:36.156854+0000","last_unstale":"2024-04-26T22:28:38.320365+0000","last_undegraded":"2024-04-26T22:28:38.320365+0000","last_fullsized":"2024-04-26T22:28:38.320365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:48.175589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319800+0000","last_change":"2024-04-26T22:28:37.167209+0000","last_active":"2024-04-26T22:28:38.319800+0000","last_peered":"2024-04-26T22:28:38.319800+0000","last_clean":"2024-04-26T22:28:38.319800+0000","last_became_active":"2024-04-26T22:28:37.166921+0000","last_became_peered":"2024-04-26T22:28:37.166921+0000","last_unstale":"2024-04-26T22:28:38.319800+0000","last_undegraded":"2024-04-26T22:28:38.319800+0000","last_fullsized":"2024-04-26T22:28:38.319800+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.605168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148815+0000","last_change":"2024-04-26T22:28:36.165408+0000","last_active":"2024-04-26T22:28:38.148815+0000","last_peered":"2024-04-26T22:28:38.148815+0000","last_clean":"2024-04-26T22:28:38.148815+0000","last_became_active":"2024-04-26T22:28:36.165110+0000","last_became_peered":"2024-04-26T22:28:36.165110+0000","last_unstale":"2024-04-26T22:28:38.148815+0000","last_undegraded":"2024-04-26T22:28:38.148815+0000","last_fullsized":"2024-04-26T22:28:38.148815+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148990+0000","last_change":"2024-04-26T22:28:37.158967+0000","last_active":"2024-04-26T22:28:38.148990+0000","last_peered":"2024-04-26T22:28:38.148990+0000","last_clean":"2024-04-26T22:28:38.148990+0000","last_became_active":"2024-04-26T22:28:37.158667+0000","last_became_peered":"2024-04-26T22:28:37.158667+0000","last_unstale":"2024-04-26T22:28:38.148990+0000","last_undegraded":"2024-04-26T22:28:38.148990+0000","last_fullsized":"2024-04-26T22:28:38.148990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:32.284968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319574+0000","last_change":"2024-04-26T22:28:36.154806+0000","last_active":"2024-04-26T22:28:38.319574+0000","last_peered":"2024-04-26T22:28:38.319574+0000","last_clean":"2024-04-26T22:28:38.319574+0000","last_became_active":"2024-04-26T22:28:36.154497+0000","last_became_peered":"2024-04-26T22:28:36.154497+0000","last_unstale":"2024-04-26T22:28:38.319574+0000","last_undegraded":"2024-04-26T22:28:38.319574+0000","last_fullsized":"2024-04-26T22:28:38.319574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319485+0000","last_change":"2024-04-26T22:28:37.156414+0000","last_active":"2024-04-26T22:28:38.319485+0000","last_peered":"2024-04-26T22:28:38.319485+0000","last_clean":"2024-04-26T22:28:38.319485+0000","last_became_active":"2024-04-26T22:28:37.156157+0000","last_became_peered":"2024-04-26T22:28:37.156157+0000","last_unstale":"2024-04-26T22:28:38.319485+0000","last_undegraded":"2024-04-26T22:28:38.319485+0000","last_fullsized":"2024-04-26T22:28:38.319485+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:37.491003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147736+0000","last_change":"2024-04-26T22:28:36.165313+0000","last_active":"2024-04-26T22:28:38.147736+0000","last_peered":"2024-04-26T22:28:38.147736+0000","last_clean":"2024-04-26T22:28:38.147736+0000","last_became_active":"2024-04-26T22:28:36.165008+0000","last_became_peered":"2024-04-26T22:28:36.165008+0000","last_unstale":"2024-04-26T22:28:38.147736+0000","last_undegraded":"2024-04-26T22:28:38.147736+0000","last_fullsized":"2024-04-26T22:28:38.147736+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148296+0000","last_change":"2024-04-26T22:28:37.155833+0000","last_active":"2024-04-26T22:28:38.148296+0000","last_peered":"2024-04-26T22:28:38.148296+0000","last_clean":"2024-04-26T22:28:38.148296+0000","last_became_active":"2024-04-26T22:28:37.155529+0000","last_became_peered":"2024-04-26T22:28:37.155529+0000","last_unstale":"2024-04-26T22:28:38.148296+0000","last_undegraded":"2024-04-26T22:28:38.148296+0000","last_fullsized":"2024-04-26T22:28:38.148296+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:58.894271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319914+0000","last_change":"2024-04-26T22:28:36.157297+0000","last_active":"2024-04-26T22:28:38.319914+0000","last_peered":"2024-04-26T22:28:38.319914+0000","last_clean":"2024-04-26T22:28:38.319914+0000","last_became_active":"2024-04-26T22:28:36.157002+0000","last_became_peered":"2024-04-26T22:28:36.157002+0000","last_unstale":"2024-04-26T22:28:38.319914+0000","last_undegraded":"2024-04-26T22:28:38.319914+0000","last_fullsized":"2024-04-26T22:28:38.319914+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:49.081592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320450+0000","last_change":"2024-04-26T22:28:37.173004+0000","last_active":"2024-04-26T22:28:38.320450+0000","last_peered":"2024-04-26T22:28:38.320450+0000","last_clean":"2024-04-26T22:28:38.320450+0000","last_became_active":"2024-04-26T22:28:37.172706+0000","last_became_peered":"2024-04-26T22:28:37.172706+0000","last_unstale":"2024-04-26T22:28:38.320450+0000","last_undegraded":"2024-04-26T22:28:38.320450+0000","last_fullsized":"2024-04-26T22:28:38.320450+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:58.429758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148075+0000","last_change":"2024-04-26T22:28:36.167783+0000","last_active":"2024-04-26T22:28:38.148075+0000","last_peered":"2024-04-26T22:28:38.148075+0000","last_clean":"2024-04-26T22:28:38.148075+0000","last_became_active":"2024-04-26T22:28:36.167480+0000","last_became_peered":"2024-04-26T22:28:36.167480+0000","last_unstale":"2024-04-26T22:28:38.148075+0000","last_undegraded":"2024-04-26T22:28:38.148075+0000","last_fullsized":"2024-04-26T22:28:38.148075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319075+0000","last_change":"2024-04-26T22:28:37.171183+0000","last_active":"2024-04-26T22:28:38.319075+0000","last_peered":"2024-04-26T22:28:38.319075+0000","last_clean":"2024-04-26T22:28:38.319075+0000","last_became_active":"2024-04-26T22:28:37.170909+0000","last_became_peered":"2024-04-26T22:28:37.170909+0000","last_unstale":"2024-04-26T22:28:38.319075+0000","last_undegraded":"2024-04-26T22:28:38.319075+0000","last_fullsized":"2024-04-26T22:28:38.319075+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:05:49.178405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148453+0000","last_change":"2024-04-26T22:28:36.161210+0000","last_active":"2024-04-26T22:28:38.148453+0000","last_peered":"2024-04-26T22:28:38.148453+0000","last_clean":"2024-04-26T22:28:38.148453+0000","last_became_active":"2024-04-26T22:28:36.160683+0000","last_became_peered":"2024-04-26T22:28:36.160683+0000","last_unstale":"2024-04-26T22:28:38.148453+0000","last_undegraded":"2024-04-26T22:28:38.148453+0000","last_fullsized":"2024-04-26T22:28:38.148453+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:56.424170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319305+0000","last_change":"2024-04-26T22:28:37.168968+0000","last_active":"2024-04-26T22:28:38.319305+0000","last_peered":"2024-04-26T22:28:38.319305+0000","last_clean":"2024-04-26T22:28:38.319305+0000","last_became_active":"2024-04-26T22:28:37.168663+0000","last_became_peered":"2024-04-26T22:28:37.168663+0000","last_unstale":"2024-04-26T22:28:38.319305+0000","last_undegraded":"2024-04-26T22:28:38.319305+0000","last_fullsized":"2024-04-26T22:28:38.319305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:57.426986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319196+0000","last_change":"2024-04-26T22:28:36.162766+0000","last_active":"2024-04-26T22:28:38.319196+0000","last_peered":"2024-04-26T22:28:38.319196+0000","last_clean":"2024-04-26T22:28:38.319196+0000","last_became_active":"2024-04-26T22:28:36.162468+0000","last_became_peered":"2024-04-26T22:28:36.162468+0000","last_unstale":"2024-04-26T22:28:38.319196+0000","last_undegraded":"2024-04-26T22:28:38.319196+0000","last_fullsized":"2024-04-26T22:28:38.319196+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:07:41.876268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148709+0000","last_change":"2024-04-26T22:28:37.167151+0000","last_active":"2024-04-26T22:28:38.148709+0000","last_peered":"2024-04-26T22:28:38.148709+0000","last_clean":"2024-04-26T22:28:38.148709+0000","last_became_active":"2024-04-26T22:28:37.166711+0000","last_became_peered":"2024-04-26T22:28:37.166711+0000","last_unstale":"2024-04-26T22:28:38.148709+0000","last_undegraded":"2024-04-26T22:28:38.148709+0000","last_fullsized":"2024-04-26T22:28:38.148709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:42.126368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148873+0000","last_change":"2024-04-26T22:28:36.162336+0000","last_active":"2024-04-26T22:28:38.148873+0000","last_peered":"2024-04-26T22:28:38.148873+0000","last_clean":"2024-04-26T22:28:38.148873+0000","last_became_active":"2024-04-26T22:28:36.162067+0000","last_became_peered":"2024-04-26T22:28:36.162067+0000","last_unstale":"2024-04-26T22:28:38.148873+0000","last_undegraded":"2024-04-26T22:28:38.148873+0000","last_fullsized":"2024-04-26T22:28:38.148873+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:56:33.358326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148357+0000","last_change":"2024-04-26T22:28:37.173777+0000","last_active":"2024-04-26T22:28:38.148357+0000","last_peered":"2024-04-26T22:28:38.148357+0000","last_clean":"2024-04-26T22:28:38.148357+0000","last_became_active":"2024-04-26T22:28:37.173494+0000","last_became_peered":"2024-04-26T22:28:37.173494+0000","last_unstale":"2024-04-26T22:28:38.148357+0000","last_undegraded":"2024-04-26T22:28:38.148357+0000","last_fullsized":"2024-04-26T22:28:38.148357+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:45.431953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320768+0000","last_change":"2024-04-26T22:28:36.161738+0000","last_active":"2024-04-26T22:28:38.320768+0000","last_peered":"2024-04-26T22:28:38.320768+0000","last_clean":"2024-04-26T22:28:38.320768+0000","last_became_active":"2024-04-26T22:28:36.161489+0000","last_became_peered":"2024-04-26T22:28:36.161489+0000","last_unstale":"2024-04-26T22:28:38.320768+0000","last_undegraded":"2024-04-26T22:28:38.320768+0000","last_fullsized":"2024-04-26T22:28:38.320768+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320053+0000","last_change":"2024-04-26T22:28:37.159609+0000","last_active":"2024-04-26T22:28:38.320053+0000","last_peered":"2024-04-26T22:28:38.320053+0000","last_clean":"2024-04-26T22:28:38.320053+0000","last_became_active":"2024-04-26T22:28:37.159355+0000","last_became_peered":"2024-04-26T22:28:37.159355+0000","last_unstale":"2024-04-26T22:28:38.320053+0000","last_undegraded":"2024-04-26T22:28:38.320053+0000","last_fullsized":"2024-04-26T22:28:38.320053+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:48.565445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148227+0000","last_change":"2024-04-26T22:28:36.159715+0000","last_active":"2024-04-26T22:28:38.148227+0000","last_peered":"2024-04-26T22:28:38.148227+0000","last_clean":"2024-04-26T22:28:38.148227+0000","last_became_active":"2024-04-26T22:28:36.159437+0000","last_became_peered":"2024-04-26T22:28:36.159437+0000","last_unstale":"2024-04-26T22:28:38.148227+0000","last_undegraded":"2024-04-26T22:28:38.148227+0000","last_fullsized":"2024-04-26T22:28:38.148227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:56:23.581702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147455+0000","last_change":"2024-04-26T22:28:37.169902+0000","last_active":"2024-04-26T22:28:38.147455+0000","last_peered":"2024-04-26T22:28:38.147455+0000","last_clean":"2024-04-26T22:28:38.147455+0000","last_became_active":"2024-04-26T22:28:37.169258+0000","last_became_peered":"2024-04-26T22:28:37.169258+0000","last_unstale":"2024-04-26T22:28:38.147455+0000","last_undegraded":"2024-04-26T22:28:38.147455+0000","last_fullsized":"2024-04-26T22:28:38.147455+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:32.284968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145646+0000","last_change":"2024-04-26T22:28:36.163369+0000","last_active":"2024-04-26T22:28:38.145646+0000","last_peered":"2024-04-26T22:28:38.145646+0000","last_clean":"2024-04-26T22:28:38.145646+0000","last_became_active":"2024-04-26T22:28:36.163060+0000","last_became_peered":"2024-04-26T22:28:36.163060+0000","last_unstale":"2024-04-26T22:28:38.145646+0000","last_undegraded":"2024-04-26T22:28:38.145646+0000","last_fullsized":"2024-04-26T22:28:38.145646+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:38.926279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319472+0000","last_change":"2024-04-26T22:28:37.163750+0000","last_active":"2024-04-26T22:28:38.319472+0000","last_peered":"2024-04-26T22:28:38.319472+0000","last_clean":"2024-04-26T22:28:38.319472+0000","last_became_active":"2024-04-26T22:28:37.163499+0000","last_became_peered":"2024-04-26T22:28:37.163499+0000","last_unstale":"2024-04-26T22:28:38.319472+0000","last_undegraded":"2024-04-26T22:28:38.319472+0000","last_fullsized":"2024-04-26T22:28:38.319472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320024+0000","last_change":"2024-04-26T22:28:36.165081+0000","last_active":"2024-04-26T22:28:38.320024+0000","last_peered":"2024-04-26T22:28:38.320024+0000","last_clean":"2024-04-26T22:28:38.320024+0000","last_became_active":"2024-04-26T22:28:36.164792+0000","last_became_peered":"2024-04-26T22:28:36.164792+0000","last_unstale":"2024-04-26T22:28:38.320024+0000","last_undegraded":"2024-04-26T22:28:38.320024+0000","last_fullsized":"2024-04-26T22:28:38.320024+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:24:42.307330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320598+0000","last_change":"2024-04-26T22:28:37.168872+0000","last_active":"2024-04-26T22:28:38.320598+0000","last_peered":"2024-04-26T22:28:38.320598+0000","last_clean":"2024-04-26T22:28:38.320598+0000","last_became_active":"2024-04-26T22:28:37.168606+0000","last_became_peered":"2024-04-26T22:28:37.168606+0000","last_unstale":"2024-04-26T22:28:38.320598+0000","last_undegraded":"2024-04-26T22:28:38.320598+0000","last_fullsized":"2024-04-26T22:28:38.320598+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:09.719373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147876+0000","last_change":"2024-04-26T22:28:36.158147+0000","last_active":"2024-04-26T22:28:38.147876+0000","last_peered":"2024-04-26T22:28:38.147876+0000","last_clean":"2024-04-26T22:28:38.147876+0000","last_became_active":"2024-04-26T22:28:36.157890+0000","last_became_peered":"2024-04-26T22:28:36.157890+0000","last_unstale":"2024-04-26T22:28:38.147876+0000","last_undegraded":"2024-04-26T22:28:38.147876+0000","last_fullsized":"2024-04-26T22:28:38.147876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:54.680123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319557+0000","last_change":"2024-04-26T22:28:37.158963+0000","last_active":"2024-04-26T22:28:38.319557+0000","last_peered":"2024-04-26T22:28:38.319557+0000","last_clean":"2024-04-26T22:28:38.319557+0000","last_became_active":"2024-04-26T22:28:37.158698+0000","last_became_peered":"2024-04-26T22:28:37.158698+0000","last_unstale":"2024-04-26T22:28:38.319557+0000","last_undegraded":"2024-04-26T22:28:38.319557+0000","last_fullsized":"2024-04-26T22:28:38.319557+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:02.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320037+0000","last_change":"2024-04-26T22:28:36.164416+0000","last_active":"2024-04-26T22:28:38.320037+0000","last_peered":"2024-04-26T22:28:38.320037+0000","last_clean":"2024-04-26T22:28:38.320037+0000","last_became_active":"2024-04-26T22:28:36.164074+0000","last_became_peered":"2024-04-26T22:28:36.164074+0000","last_unstale":"2024-04-26T22:28:38.320037+0000","last_undegraded":"2024-04-26T22:28:38.320037+0000","last_fullsized":"2024-04-26T22:28:38.320037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:15.945211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147661+0000","last_change":"2024-04-26T22:28:37.170711+0000","last_active":"2024-04-26T22:28:38.147661+0000","last_peered":"2024-04-26T22:28:38.147661+0000","last_clean":"2024-04-26T22:28:38.147661+0000","last_became_active":"2024-04-26T22:28:37.170248+0000","last_became_peered":"2024-04-26T22:28:37.170248+0000","last_unstale":"2024-04-26T22:28:38.147661+0000","last_undegraded":"2024-04-26T22:28:38.147661+0000","last_fullsized":"2024-04-26T22:28:38.147661+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:02.822292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320164+0000","last_change":"2024-04-26T22:28:36.167663+0000","last_active":"2024-04-26T22:28:38.320164+0000","last_peered":"2024-04-26T22:28:38.320164+0000","last_clean":"2024-04-26T22:28:38.320164+0000","last_became_active":"2024-04-26T22:28:36.167430+0000","last_became_peered":"2024-04-26T22:28:36.167430+0000","last_unstale":"2024-04-26T22:28:38.320164+0000","last_undegraded":"2024-04-26T22:28:38.320164+0000","last_fullsized":"2024-04-26T22:28:38.320164+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145603+0000","last_change":"2024-04-26T22:28:37.163207+0000","last_active":"2024-04-26T22:28:38.145603+0000","last_peered":"2024-04-26T22:28:38.145603+0000","last_clean":"2024-04-26T22:28:38.145603+0000","last_became_active":"2024-04-26T22:28:37.162935+0000","last_became_peered":"2024-04-26T22:28:37.162935+0000","last_unstale":"2024-04-26T22:28:38.145603+0000","last_undegraded":"2024-04-26T22:28:38.145603+0000","last_fullsized":"2024-04-26T22:28:38.145603+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:42.001465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145625+0000","last_change":"2024-04-26T22:28:36.152997+0000","last_active":"2024-04-26T22:28:38.145625+0000","last_peered":"2024-04-26T22:28:38.145625+0000","last_clean":"2024-04-26T22:28:38.145625+0000","last_became_active":"2024-04-26T22:28:36.152694+0000","last_became_peered":"2024-04-26T22:28:36.152694+0000","last_unstale":"2024-04-26T22:28:38.145625+0000","last_undegraded":"2024-04-26T22:28:38.145625+0000","last_fullsized":"2024-04-26T22:28:38.145625+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:28:10.294547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320843+0000","last_change":"2024-04-26T22:28:37.170076+0000","last_active":"2024-04-26T22:28:38.320843+0000","last_peered":"2024-04-26T22:28:38.320843+0000","last_clean":"2024-04-26T22:28:38.320843+0000","last_became_active":"2024-04-26T22:28:37.169231+0000","last_became_peered":"2024-04-26T22:28:37.169231+0000","last_unstale":"2024-04-26T22:28:38.320843+0000","last_undegraded":"2024-04-26T22:28:38.320843+0000","last_fullsized":"2024-04-26T22:28:38.320843+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:02.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148788+0000","last_change":"2024-04-26T22:28:36.162940+0000","last_active":"2024-04-26T22:28:38.148788+0000","last_peered":"2024-04-26T22:28:38.148788+0000","last_clean":"2024-04-26T22:28:38.148788+0000","last_became_active":"2024-04-26T22:28:36.162676+0000","last_became_peered":"2024-04-26T22:28:36.162676+0000","last_unstale":"2024-04-26T22:28:38.148788+0000","last_undegraded":"2024-04-26T22:28:38.148788+0000","last_fullsized":"2024-04-26T22:28:38.148788+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319466+0000","last_change":"2024-04-26T22:28:37.164222+0000","last_active":"2024-04-26T22:28:38.319466+0000","last_peered":"2024-04-26T22:28:38.319466+0000","last_clean":"2024-04-26T22:28:38.319466+0000","last_became_active":"2024-04-26T22:28:37.163891+0000","last_became_peered":"2024-04-26T22:28:37.163891+0000","last_unstale":"2024-04-26T22:28:38.319466+0000","last_undegraded":"2024-04-26T22:28:38.319466+0000","last_fullsized":"2024-04-26T22:28:38.319466+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:08.248999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147497+0000","last_change":"2024-04-26T22:28:36.157167+0000","last_active":"2024-04-26T22:28:38.147497+0000","last_peered":"2024-04-26T22:28:38.147497+0000","last_clean":"2024-04-26T22:28:38.147497+0000","last_became_active":"2024-04-26T22:28:36.156867+0000","last_became_peered":"2024-04-26T22:28:36.156867+0000","last_unstale":"2024-04-26T22:28:38.147497+0000","last_undegraded":"2024-04-26T22:28:38.147497+0000","last_fullsized":"2024-04-26T22:28:38.147497+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:47.426412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319697+0000","last_change":"2024-04-26T22:28:37.166273+0000","last_active":"2024-04-26T22:28:38.319697+0000","last_peered":"2024-04-26T22:28:38.319697+0000","last_clean":"2024-04-26T22:28:38.319697+0000","last_became_active":"2024-04-26T22:28:37.166001+0000","last_became_peered":"2024-04-26T22:28:37.166001+0000","last_unstale":"2024-04-26T22:28:38.319697+0000","last_undegraded":"2024-04-26T22:28:38.319697+0000","last_fullsized":"2024-04-26T22:28:38.319697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:45.781348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320100+0000","last_change":"2024-04-26T22:28:36.160489+0000","last_active":"2024-04-26T22:28:38.320100+0000","last_peered":"2024-04-26T22:28:38.320100+0000","last_clean":"2024-04-26T22:28:38.320100+0000","last_became_active":"2024-04-26T22:28:36.160227+0000","last_became_peered":"2024-04-26T22:28:36.160227+0000","last_unstale":"2024-04-26T22:28:38.320100+0000","last_undegraded":"2024-04-26T22:28:38.320100+0000","last_fullsized":"2024-04-26T22:28:38.320100+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:15:10.194762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319853+0000","last_change":"2024-04-26T22:28:37.163484+0000","last_active":"2024-04-26T22:28:38.319853+0000","last_peered":"2024-04-26T22:28:38.319853+0000","last_clean":"2024-04-26T22:28:38.319853+0000","last_became_active":"2024-04-26T22:28:37.163149+0000","last_became_peered":"2024-04-26T22:28:37.163149+0000","last_unstale":"2024-04-26T22:28:38.319853+0000","last_undegraded":"2024-04-26T22:28:38.319853+0000","last_fullsized":"2024-04-26T22:28:38.319853+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:32.473533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320150+0000","last_change":"2024-04-26T22:28:36.156431+0000","last_active":"2024-04-26T22:28:38.320150+0000","last_peered":"2024-04-26T22:28:38.320150+0000","last_clean":"2024-04-26T22:28:38.320150+0000","last_became_active":"2024-04-26T22:28:36.156129+0000","last_became_peered":"2024-04-26T22:28:36.156129+0000","last_unstale":"2024-04-26T22:28:38.320150+0000","last_undegraded":"2024-04-26T22:28:38.320150+0000","last_fullsized":"2024-04-26T22:28:38.320150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:41.123552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319477+0000","last_change":"2024-04-26T22:28:37.153551+0000","last_active":"2024-04-26T22:28:38.319477+0000","last_peered":"2024-04-26T22:28:38.319477+0000","last_clean":"2024-04-26T22:28:38.319477+0000","last_became_active":"2024-04-26T22:28:37.152598+0000","last_became_peered":"2024-04-26T22:28:37.152598+0000","last_unstale":"2024-04-26T22:28:38.319477+0000","last_undegraded":"2024-04-26T22:28:38.319477+0000","last_fullsized":"2024-04-26T22:28:38.319477+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:52.225117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148499+0000","last_change":"2024-04-26T22:28:36.156716+0000","last_active":"2024-04-26T22:28:38.148499+0000","last_peered":"2024-04-26T22:28:38.148499+0000","last_clean":"2024-04-26T22:28:38.148499+0000","last_became_active":"2024-04-26T22:28:36.156424+0000","last_became_peered":"2024-04-26T22:28:36.156424+0000","last_unstale":"2024-04-26T22:28:38.148499+0000","last_undegraded":"2024-04-26T22:28:38.148499+0000","last_fullsized":"2024-04-26T22:28:38.148499+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:36:07.246183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.145384+0000","last_change":"2024-04-26T22:28:37.167506+0000","last_active":"2024-04-26T22:28:38.145384+0000","last_peered":"2024-04-26T22:28:38.145384+0000","last_clean":"2024-04-26T22:28:38.145384+0000","last_became_active":"2024-04-26T22:28:37.167182+0000","last_became_peered":"2024-04-26T22:28:37.167182+0000","last_unstale":"2024-04-26T22:28:38.145384+0000","last_undegraded":"2024-04-26T22:28:38.145384+0000","last_fullsized":"2024-04-26T22:28:38.145384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:25:28.175186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320632+0000","last_change":"2024-04-26T22:28:36.157532+0000","last_active":"2024-04-26T22:28:38.320632+0000","last_peered":"2024-04-26T22:28:38.320632+0000","last_clean":"2024-04-26T22:28:38.320632+0000","last_became_active":"2024-04-26T22:28:36.157238+0000","last_became_peered":"2024-04-26T22:28:36.157238+0000","last_unstale":"2024-04-26T22:28:38.320632+0000","last_undegraded":"2024-04-26T22:28:38.320632+0000","last_fullsized":"2024-04-26T22:28:38.320632+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:45.026044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320575+0000","last_change":"2024-04-26T22:28:37.164428+0000","last_active":"2024-04-26T22:28:38.320575+0000","last_peered":"2024-04-26T22:28:38.320575+0000","last_clean":"2024-04-26T22:28:38.320575+0000","last_became_active":"2024-04-26T22:28:37.164171+0000","last_became_peered":"2024-04-26T22:28:37.164171+0000","last_unstale":"2024-04-26T22:28:38.320575+0000","last_undegraded":"2024-04-26T22:28:38.320575+0000","last_fullsized":"2024-04-26T22:28:38.320575+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:39:45.431953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147188+0000","last_change":"2024-04-26T22:28:36.157860+0000","last_active":"2024-04-26T22:28:38.147188+0000","last_peered":"2024-04-26T22:28:38.147188+0000","last_clean":"2024-04-26T22:28:38.147188+0000","last_became_active":"2024-04-26T22:28:36.157588+0000","last_became_peered":"2024-04-26T22:28:36.157588+0000","last_unstale":"2024-04-26T22:28:38.147188+0000","last_undegraded":"2024-04-26T22:28:38.147188+0000","last_fullsized":"2024-04-26T22:28:38.147188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:18.602353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319900+0000","last_change":"2024-04-26T22:28:37.162836+0000","last_active":"2024-04-26T22:28:38.319900+0000","last_peered":"2024-04-26T22:28:38.319900+0000","last_clean":"2024-04-26T22:28:38.319900+0000","last_became_active":"2024-04-26T22:28:37.162524+0000","last_became_peered":"2024-04-26T22:28:37.162524+0000","last_unstale":"2024-04-26T22:28:38.319900+0000","last_undegraded":"2024-04-26T22:28:38.319900+0000","last_fullsized":"2024-04-26T22:28:38.319900+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:50.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320178+0000","last_change":"2024-04-26T22:28:36.165811+0000","last_active":"2024-04-26T22:28:38.320178+0000","last_peered":"2024-04-26T22:28:38.320178+0000","last_clean":"2024-04-26T22:28:38.320178+0000","last_became_active":"2024-04-26T22:28:36.165461+0000","last_became_peered":"2024-04-26T22:28:36.165461+0000","last_unstale":"2024-04-26T22:28:38.320178+0000","last_undegraded":"2024-04-26T22:28:38.320178+0000","last_fullsized":"2024-04-26T22:28:38.320178+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:30:31.282152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320210+0000","last_change":"2024-04-26T22:28:37.169916+0000","last_active":"2024-04-26T22:28:38.320210+0000","last_peered":"2024-04-26T22:28:38.320210+0000","last_clean":"2024-04-26T22:28:38.320210+0000","last_became_active":"2024-04-26T22:28:37.169635+0000","last_became_peered":"2024-04-26T22:28:37.169635+0000","last_unstale":"2024-04-26T22:28:38.320210+0000","last_undegraded":"2024-04-26T22:28:38.320210+0000","last_fullsized":"2024-04-26T22:28:38.320210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:16.948027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148325+0000","last_change":"2024-04-26T22:28:36.159775+0000","last_active":"2024-04-26T22:28:38.148325+0000","last_peered":"2024-04-26T22:28:38.148325+0000","last_clean":"2024-04-26T22:28:38.148325+0000","last_became_active":"2024-04-26T22:28:36.159490+0000","last_became_peered":"2024-04-26T22:28:36.159490+0000","last_unstale":"2024-04-26T22:28:38.148325+0000","last_undegraded":"2024-04-26T22:28:38.148325+0000","last_fullsized":"2024-04-26T22:28:38.148325+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:50:44.778533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147630+0000","last_change":"2024-04-26T22:28:37.171354+0000","last_active":"2024-04-26T22:28:38.147630+0000","last_peered":"2024-04-26T22:28:38.147630+0000","last_clean":"2024-04-26T22:28:38.147630+0000","last_became_active":"2024-04-26T22:28:37.169646+0000","last_became_peered":"2024-04-26T22:28:37.169646+0000","last_unstale":"2024-04-26T22:28:38.147630+0000","last_undegraded":"2024-04-26T22:28:38.147630+0000","last_fullsized":"2024-04-26T22:28:38.147630+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:17:41.641104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319277+0000","last_change":"2024-04-26T22:28:36.154743+0000","last_active":"2024-04-26T22:28:38.319277+0000","last_peered":"2024-04-26T22:28:38.319277+0000","last_clean":"2024-04-26T22:28:38.319277+0000","last_became_active":"2024-04-26T22:28:36.154433+0000","last_became_peered":"2024-04-26T22:28:36.154433+0000","last_unstale":"2024-04-26T22:28:38.319277+0000","last_undegraded":"2024-04-26T22:28:38.319277+0000","last_fullsized":"2024-04-26T22:28:38.319277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:40.391689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319999+0000","last_change":"2024-04-26T22:28:37.156882+0000","last_active":"2024-04-26T22:28:38.319999+0000","last_peered":"2024-04-26T22:28:38.319999+0000","last_clean":"2024-04-26T22:28:38.319999+0000","last_became_active":"2024-04-26T22:28:37.156555+0000","last_became_peered":"2024-04-26T22:28:37.156555+0000","last_unstale":"2024-04-26T22:28:38.319999+0000","last_undegraded":"2024-04-26T22:28:38.319999+0000","last_fullsized":"2024-04-26T22:28:38.319999+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:29.019432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147607+0000","last_change":"2024-04-26T22:28:36.160465+0000","last_active":"2024-04-26T22:28:38.147607+0000","last_peered":"2024-04-26T22:28:38.147607+0000","last_clean":"2024-04-26T22:28:38.147607+0000","last_became_active":"2024-04-26T22:28:36.160206+0000","last_became_peered":"2024-04-26T22:28:36.160206+0000","last_unstale":"2024-04-26T22:28:38.147607+0000","last_undegraded":"2024-04-26T22:28:38.147607+0000","last_fullsized":"2024-04-26T22:28:38.147607+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:53:36.488188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319824+0000","last_change":"2024-04-26T22:28:37.157525+0000","last_active":"2024-04-26T22:28:38.319824+0000","last_peered":"2024-04-26T22:28:38.319824+0000","last_clean":"2024-04-26T22:28:38.319824+0000","last_became_active":"2024-04-26T22:28:37.157223+0000","last_became_peered":"2024-04-26T22:28:37.157223+0000","last_unstale":"2024-04-26T22:28:38.319824+0000","last_undegraded":"2024-04-26T22:28:38.319824+0000","last_fullsized":"2024-04-26T22:28:38.319824+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:38:56.903638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148195+0000","last_change":"2024-04-26T22:28:36.163006+0000","last_active":"2024-04-26T22:28:38.148195+0000","last_peered":"2024-04-26T22:28:38.148195+0000","last_clean":"2024-04-26T22:28:38.148195+0000","last_became_active":"2024-04-26T22:28:36.162735+0000","last_became_peered":"2024-04-26T22:28:36.162735+0000","last_unstale":"2024-04-26T22:28:38.148195+0000","last_undegraded":"2024-04-26T22:28:38.148195+0000","last_fullsized":"2024-04-26T22:28:38.148195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:13.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.148035+0000","last_change":"2024-04-26T22:28:37.157113+0000","last_active":"2024-04-26T22:28:38.148035+0000","last_peered":"2024-04-26T22:28:38.148035+0000","last_clean":"2024-04-26T22:28:38.148035+0000","last_became_active":"2024-04-26T22:28:37.156837+0000","last_became_peered":"2024-04-26T22:28:37.156837+0000","last_unstale":"2024-04-26T22:28:38.148035+0000","last_undegraded":"2024-04-26T22:28:38.148035+0000","last_fullsized":"2024-04-26T22:28:38.148035+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:50.084408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.319205+0000","last_change":"2024-04-26T22:28:36.159382+0000","last_active":"2024-04-26T22:28:38.319205+0000","last_peered":"2024-04-26T22:28:38.319205+0000","last_clean":"2024-04-26T22:28:38.319205+0000","last_became_active":"2024-04-26T22:28:36.159016+0000","last_became_peered":"2024-04-26T22:28:36.159016+0000","last_unstale":"2024-04-26T22:28:38.319205+0000","last_undegraded":"2024-04-26T22:28:38.319205+0000","last_fullsized":"2024-04-26T22:28:38.319205+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147922+0000","last_change":"2024-04-26T22:28:37.164624+0000","last_active":"2024-04-26T22:28:38.147922+0000","last_peered":"2024-04-26T22:28:38.147922+0000","last_clean":"2024-04-26T22:28:38.147922+0000","last_became_active":"2024-04-26T22:28:37.164175+0000","last_became_peered":"2024-04-26T22:28:37.164175+0000","last_unstale":"2024-04-26T22:28:38.147922+0000","last_undegraded":"2024-04-26T22:28:38.147922+0000","last_fullsized":"2024-04-26T22:28:38.147922+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:19.605168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320277+0000","last_change":"2024-04-26T22:28:36.162470+0000","last_active":"2024-04-26T22:28:38.320277+0000","last_peered":"2024-04-26T22:28:38.320277+0000","last_clean":"2024-04-26T22:28:38.320277+0000","last_became_active":"2024-04-26T22:28:36.162164+0000","last_became_peered":"2024-04-26T22:28:36.162164+0000","last_unstale":"2024-04-26T22:28:38.320277+0000","last_undegraded":"2024-04-26T22:28:38.320277+0000","last_fullsized":"2024-04-26T22:28:38.320277+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:15.467365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.320680+0000","last_change":"2024-04-26T22:28:37.171688+0000","last_active":"2024-04-26T22:28:38.320680+0000","last_peered":"2024-04-26T22:28:38.320680+0000","last_clean":"2024-04-26T22:28:38.320680+0000","last_became_active":"2024-04-26T22:28:37.170925+0000","last_became_peered":"2024-04-26T22:28:37.170925+0000","last_unstale":"2024-04-26T22:28:38.320680+0000","last_undegraded":"2024-04-26T22:28:38.320680+0000","last_fullsized":"2024-04-26T22:28:38.320680+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:36.133205+0000","last_clean_scrub_stamp":"2024-04-26T22:28:36.133205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:57:24.557002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147782+0000","last_change":"2024-04-26T22:28:36.162279+0000","last_active":"2024-04-26T22:28:38.147782+0000","last_peered":"2024-04-26T22:28:38.147782+0000","last_clean":"2024-04-26T22:28:38.147782+0000","last_became_active":"2024-04-26T22:28:36.161730+0000","last_became_peered":"2024-04-26T22:28:36.161730+0000","last_unstale":"2024-04-26T22:28:38.147782+0000","last_undegraded":"2024-04-26T22:28:38.147782+0000","last_fullsized":"2024-04-26T22:28:38.147782+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:28:35.130390+0000","last_clean_scrub_stamp":"2024-04-26T22:28:35.130390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:51.222302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T22:28:38.147329+0000","last_change":"2024-04-26T22:27:03.724503+0000","last_active":"2024-04-26T22:28:38.147329+0000","last_peered":"2024-04-26T22:28:38.147329+0000","last_clean":"2024-04-26T22:28:38.147329+0000","last_became_active":"2024-04-26T22:27:03.723529+0000","last_became_peered":"2024-04-26T22:27:03.723529+0000","last_unstale":"2024-04-26T22:28:38.147329+0000","last_undegraded":"2024-04-26T22:28:38.147329+0000","last_fullsized":"2024-04-26T22:28:38.147329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T22:27:01.755385+0000","last_clean_scrub_stamp":"2024-04-26T22:27:01.755385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:22.885091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":793656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":203870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":204810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":203836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":204832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":507904,"data_stored":203268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1101824,"data_stored":793600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T22:28:44.942 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T22:28:44.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T22:28:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.085+0000 7f2fa7904640 1 Processor -- start 2024-04-26T22:28:45.085 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.085+0000 7f2fa7904640 1 -- start start 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa7904640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 0x7f2fa00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa7904640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fa00d1fe0 0x7f2fa00d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa7904640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa00cd640 con 0x7f2fa00b2290 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa7904640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa00cd920 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa7904640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00cdc00 con 0x7f2fa00cff80 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa5ea1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fa00d1fe0 0x7f2fa00d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa5ea1640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f2fa00d1fe0 0x7f2fa00d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38134/0 (socket says 172.21.15.62:38134) 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa5ea1640 1 -- 172.21.15.62:0/2636750070 learned_addr learned my addr 172.21.15.62:0/2636750070 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.086+0000 7f2fa4e9f640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 0x7f2fa00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.087 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa5ea1640 1 -- 172.21.15.62:0/2636750070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 msgr2=0x7f2fa00b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa5ea1640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 0x7f2fa00b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa5ea1640 1 -- 172.21.15.62:0/2636750070 >> v1:172.21.15.174:6789/0 conn(0x7f2fa00b2290 legacy=0x7f2fa00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa5ea1640 1 -- 172.21.15.62:0/2636750070 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00cdee0 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.088 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa4e9f640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 0x7f2fa00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2fa5ea1640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00d2450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009870 tx=0x7f2f9002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094c7ae4d310b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.087+0000 7f2f9ffff640 1 -- 172.21.15.62:0/2636750070 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90037070 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2f9ffff640 1 -- 172.21.15.62:0/2636750070 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90032440 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2f9ffff640 1 -- 172.21.15.62:0/2636750070 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90032810 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.089 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 -- 172.21.15.62:0/2636750070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00d1fe0 msgr2=0x7f2fa00d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00d2450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009870 tx=0x7f2f9002a250 comp rx=0 tx=0).stop 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 -- 172.21.15.62:0/2636750070 shutdown_connections 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00d2450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 --2- 172.21.15.62:0/2636750070 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00cff80 0x7f2fa00b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 -- 172.21.15.62:0/2636750070 >> 172.21.15.62:0/2636750070 conn(0x7f2fa00d2ff0 msgr2=0x7f2fa00d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 -- 172.21.15.62:0/2636750070 shutdown_connections 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.088+0000 7f2fa7904640 1 -- 172.21.15.62:0/2636750070 wait complete. 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.089+0000 7f2fa7904640 1 Processor -- start 2024-04-26T22:28:45.090 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.089+0000 7f2fa7904640 1 -- start start 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00b2290 0x7f2fa00b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00d2990 con 0x7f2fa00cff80 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00b2670 con 0x7f2fa00d1fe0 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa7904640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa00d2b10 con 0x7f2fa00b2290 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa4e9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.092 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa4e9f640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38140/0 (socket says 172.21.15.62:38140) 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa4e9f640 1 -- 172.21.15.62:0/3605444178 learned_addr learned my addr 172.21.15.62:0/3605444178 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa56a0640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00b2290 0x7f2fa00b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa5ea1640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa4e9f640 1 -- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00b2290 msgr2=0x7f2fa00b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.090+0000 7f2fa4e9f640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00b2290 0x7f2fa00b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa4e9f640 1 -- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa00d1fe0 msgr2=0x7f2fa00b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa4e9f640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa4e9f640 1 -- 172.21.15.62:0/3605444178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00b6890 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa4e9f640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e760 tx=0x7f2f9400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400f040 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f94004230 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400ee70 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa00b6b70 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.091+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa00b7130 con 0x7f2fa00cff80 2024-04-26T22:28:45.093 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.092+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa00b4850 con 0x7f2fa00cff80 2024-04-26T22:28:45.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.092+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f2f9400c6d0 con 0x7f2fa00cff80 2024-04-26T22:28:45.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.093+0000 7f2f9dffb640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 0x7f2f8c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.094 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.093+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2f9408b860 con 0x7f2fa00cff80 2024-04-26T22:28:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.093+0000 7f2fa56a0640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 0x7f2f8c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.095 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.094+0000 7f2fa56a0640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 0x7f2f8c06ee70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001f10 tx=0x7f2f98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.097 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.097+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fa00b4850 con 0x7f2fa00cff80 2024-04-26T22:28:45.359 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.359+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2fa00ce9c0 con 0x7f2fa00cff80 2024-04-26T22:28:45.361 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.361+0000 7f2f9dffb640 1 -- 172.21.15.62:0/3605444178 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f2f94017020 con 0x7f2fa00cff80 2024-04-26T22:28:45.361 INFO:teuthology.orchestra.run.smithi062.stdout:[client.0] 2024-04-26T22:28:45.361 INFO:teuthology.orchestra.run.smithi062.stdout: key = AQAIKixmq9blFBAAeH7q1eGp3+LduEwWkpiKlQ== 2024-04-26T22:28:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: caps mds = "allow" 2024-04-26T22:28:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: caps mon = "allow r" 2024-04-26T22:28:45.362 INFO:teuthology.orchestra.run.smithi062.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T22:28:45.362 INFO:teuthology.orchestra.run.smithi062.stderr:updated caps for client.0 2024-04-26T22:28:45.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 msgr2=0x7f2f8c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 0x7f2f8c06ee70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f98001f10 tx=0x7f2f98009040 comp rx=0 tx=0).stop 2024-04-26T22:28:45.364 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 msgr2=0x7f2fa0156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400e760 tx=0x7f2f9400ec20 comp rx=0 tx=0).stop 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 shutdown_connections 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f2f8c06c9b0 0x7f2f8c06ee70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa00d1fe0 0x7f2fa00b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2fa00cff80 0x7f2fa0156300 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 --2- 172.21.15.62:0/3605444178 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2fa00b2290 0x7f2fa00b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 >> 172.21.15.62:0/3605444178 conn(0x7f2fa00d2ff0 msgr2=0x7f2fa00d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 shutdown_connections 2024-04-26T22:28:45.365 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.364+0000 7f2fa7904640 1 -- 172.21.15.62:0/3605444178 wait complete. 2024-04-26T22:28:45.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T22:28:45.475 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.475+0000 7f52817f0640 1 Processor -- start 2024-04-26T22:28:45.476 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.475+0000 7f52817f0640 1 -- start start 2024-04-26T22:28:45.477 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 0x7f527c0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f527c0d3820 0x7f527c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c0d1850 0x7f527c0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f527c0d41c0 con 0x7f527c0d3820 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f527c0d0210 con 0x7f527c162b40 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f52817f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f527c0d04f0 con 0x7f527c0d1850 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f5273fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f527c0d3820 0x7f527c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f5273fff640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f527c0d3820 0x7f527c0d3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38148/0 (socket says 172.21.15.62:38148) 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.476+0000 7f5273fff640 1 -- 172.21.15.62:0/250870863 learned_addr learned my addr 172.21.15.62:0/250870863 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f527b7fe640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c0d1850 0x7f527c0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f527affd640 1 --2- 172.21.15.62:0/250870863 >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 0x7f527c0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 -- 172.21.15.62:0/250870863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c0d1850 msgr2=0x7f527c0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:45.478 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c0d1850 0x7f527c0d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 -- 172.21.15.62:0/250870863 >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 msgr2=0x7f527c0d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 --2- 172.21.15.62:0/250870863 >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 0x7f527c0d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 -- 172.21.15.62:0/250870863 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c0d07d0 con 0x7f527c0d3820 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f527affd640 1 --2- 172.21.15.62:0/250870863 >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 0x7f527c0d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.477+0000 7f5273fff640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0d3c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f527c0d2910 tx=0x7f526402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e860278660b034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f527a7fc640 1 -- 172.21.15.62:0/250870863 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264037070 con 0x7f527c0d3820 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f527a7fc640 1 -- 172.21.15.62:0/250870863 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5264032440 con 0x7f527c0d3820 2024-04-26T22:28:45.479 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f527a7fc640 1 -- 172.21.15.62:0/250870863 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264032810 con 0x7f527c0d3820 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 -- 172.21.15.62:0/250870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 msgr2=0x7f527c0d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0d3c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f527c0d2910 tx=0x7f526402a270 comp rx=0 tx=0).stop 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 -- 172.21.15.62:0/250870863 shutdown_connections 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c0d1850 0x7f527c0d1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 --2- 172.21.15.62:0/250870863 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0d3c80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 --2- 172.21.15.62:0/250870863 >> v2:172.21.15.174:3300/0 conn(0x7f527c162b40 0x7f527c0d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 -- 172.21.15.62:0/250870863 >> 172.21.15.62:0/250870863 conn(0x7f527c09e550 msgr2=0x7f527c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 -- 172.21.15.62:0/250870863 shutdown_connections 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.478+0000 7f52817f0640 1 -- 172.21.15.62:0/250870863 wait complete. 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.479+0000 7f52817f0640 1 Processor -- start 2024-04-26T22:28:45.482 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.479+0000 7f52817f0640 1 -- start start 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c0d1850 0x7f527c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c162b40 0x7f527c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f527c0d6110 con 0x7f527c0d3820 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f527c0d0d90 con 0x7f527c0d1850 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f52817f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f527c0d12b0 con 0x7f527c162b40 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38152/0 (socket says 172.21.15.62:38152) 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 -- 172.21.15.62:0/1525589174 learned_addr learned my addr 172.21.15.62:0/1525589174 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f527b7fe640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c162b40 0x7f527c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f527affd640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c0d1850 0x7f527c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 -- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c162b40 msgr2=0x7f527c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c162b40 0x7f527c0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.483 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 -- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c0d1850 msgr2=0x7f527c1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c0d1850 0x7f527c1516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.480+0000 7f5273fff640 1 -- 172.21.15.62:0/1525589174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c15df80 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f5273fff640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f526402a2b0 tx=0x7f52640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264041020 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52640306c0 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c151be0 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264037040 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.481+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c0c9790 con 0x7f527c0d3820 2024-04-26T22:28:45.484 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.482+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527c06a360 con 0x7f527c0d3820 2024-04-26T22:28:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.482+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5264044020 con 0x7f527c0d3820 2024-04-26T22:28:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.483+0000 7f52737fe640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 0x7f525c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.483+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f52640ac2b0 con 0x7f527c0d3820 2024-04-26T22:28:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.483+0000 7f527affd640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 0x7f525c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:45.485 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.483+0000 7f527affd640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 0x7f525c06ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f526c00a990 tx=0x7f526c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:45.487 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.487+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f526404b030 con 0x7f527c0d3820 2024-04-26T22:28:45.713 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.713+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f527c0d2960 con 0x7f527c0d3820 2024-04-26T22:28:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.714+0000 7f52737fe640 1 -- 172.21.15.62:0/1525589174 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5187 (secure 0 0 0) 0x7f527c151be0 con 0x7f527c0d3820 2024-04-26T22:28:45.714 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-26T22:28:45.714 INFO:teuthology.orchestra.run.smithi062.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5012,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1398555464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6835","nonce":1398555464},{"type":"v1","addr":"172.21.15.174:6837","nonce":1398555464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5544,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6834/1896156282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1896156282},{"type":"v1","addr":"172.21.15.62:6834","nonce":1896156282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3059612747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6833","nonce":3059612747},{"type":"v1","addr":"172.21.15.62:6835","nonce":3059612747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T22:28:36.514552+0000","modified":"2024-04-26T22:28:38.513685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5559},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5559":{"gid":5559,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6836/4128033051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":4128033051},{"type":"v1","addr":"172.21.15.174:6836","nonce":4128033051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5559,"qdb_cluster":[5559]},"id":3}]} 2024-04-26T22:28:45.714 INFO:teuthology.orchestra.run.smithi062.stderr:dumped fsmap epoch 23 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 msgr2=0x7f525c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 0x7f525c06ee40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f526c00a990 tx=0x7f526c008040 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 msgr2=0x7f527c0ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f526402a2b0 tx=0x7f52640046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 shutdown_connections 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f525c06c980 0x7f525c06ee40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f527c162b40 0x7f527c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f527c0d3820 0x7f527c0ce5b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 --2- 172.21.15.62:0/1525589174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c0d1850 0x7f527c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:45.716 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 >> 172.21.15.62:0/1525589174 conn(0x7f527c09e550 msgr2=0x7f527c0b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 shutdown_connections 2024-04-26T22:28:45.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:45.716+0000 7f52817f0640 1 -- 172.21.15.62:0/1525589174 wait complete. 2024-04-26T22:28:46.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T22:28:36.514552+0000', 'modified': '2024-04-26T22:28:38.513685+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5559}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5559': {'gid': 5559, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6836/4128033051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 4128033051}, {'type': 'v1', 'addr': '172.21.15.174:6836', 'nonce': 4128033051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5559, 'qdb_cluster': [5559]} 2024-04-26T22:28:46.130 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T22:28:46.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T22:28:46.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.235+0000 7fac9bce1640 1 Processor -- start 2024-04-26T22:28:46.235 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.235+0000 7fac9bce1640 1 -- start start 2024-04-26T22:28:46.237 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9bce1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac940d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9bce1640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac940b1d70 con 0x7fac940b2050 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9bce1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac940d1960 con 0x7fac940cff40 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9bce1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fac94163220 con 0x7fac94162e40 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9927c640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fac940b2050 0x7fac940cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49856/0 (socket says 172.21.15.62:49856) 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9927c640 1 -- 172.21.15.62:0/1007601477 learned_addr learned my addr 172.21.15.62:0/1007601477 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac99a7d640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac940d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219583087 0 0) 0x7fac940b1d70 con 0x7fac940b2050 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac80003520 con 0x7fac940b2050 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1795408820 0 0) 0x7fac80003520 con 0x7fac940b2050 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 msgr2=0x7fac940d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac940d1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 >> v1:172.21.15.174:6789/0 conn(0x7fac940cff40 legacy=0x7fac940ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.238 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac940d1c40 con 0x7fac940b2050 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fac84001670 con 0x7fac940b2050 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 >> v1:172.21.15.62:6789/0 conn(0x7fac940b2050 legacy=0x7fac940cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac800039e0 0x7fac80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac8000bac0 con 0x7fac80004800 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac8000bc70 con 0x7fac800039e0 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fac8000be70 con 0x7fac94162e40 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.236+0000 7fac9927c640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac99a7d640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac800039e0 0x7fac80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9a27e640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 -- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 msgr2=0x7fac80005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac94162e40 0x7fac80005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 -- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac800039e0 msgr2=0x7fac80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.239 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac800039e0 0x7fac80003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 -- 172.21.15.62:0/1007601477 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac80003520 con 0x7fac80004800 2024-04-26T22:28:46.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.237+0000 7fac9927c640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac80004c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c210 tx=0x7fac80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7447231e03b0f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84007d30 con 0x7fac80004800 2024-04-26T22:28:46.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84005ad0 con 0x7fac80004800 2024-04-26T22:28:46.240 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac98a7b640 1 -- 172.21.15.62:0/1007601477 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84007070 con 0x7fac80004800 2024-04-26T22:28:46.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 -- 172.21.15.62:0/1007601477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 msgr2=0x7fac80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 --2- 172.21.15.62:0/1007601477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac80004c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c210 tx=0x7fac80005830 comp rx=0 tx=0).stop 2024-04-26T22:28:46.242 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9a27e640 1 -- 172.21.15.62:0/1007601477 reap_dead start 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 -- 172.21.15.62:0/1007601477 shutdown_connections 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 -- 172.21.15.62:0/1007601477 >> 172.21.15.62:0/1007601477 conn(0x7fac9409ba50 msgr2=0x7fac9409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 -- 172.21.15.62:0/1007601477 shutdown_connections 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.238+0000 7fac9bce1640 1 -- 172.21.15.62:0/1007601477 wait complete. 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.239+0000 7fac9bce1640 1 Processor -- start 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 -- start start 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac940caa60 0x7fac940caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac9415ac70 0x7fac9415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac940b24b0 con 0x7fac80004800 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac940d0eb0 con 0x7fac940caa60 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac9bce1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fac940b2630 con 0x7fac9415ac70 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac99a7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.243 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac99a7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38168/0 (socket says 172.21.15.62:38168) 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.240+0000 7fac99a7d640 1 -- 172.21.15.62:0/4117093468 learned_addr learned my addr 172.21.15.62:0/4117093468 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac9927c640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac940caa60 0x7fac940caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac9a27e640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac9415ac70 0x7fac9415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 -- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac9415ac70 msgr2=0x7fac9415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac9415ac70 0x7fac9415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 -- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac940caa60 msgr2=0x7fac940caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac940caa60 0x7fac940caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 -- 172.21.15.62:0/4117093468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9415aa70 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac99a7d640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fac90011a00 tx=0x7fac90011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac8a7fc640 1 -- 172.21.15.62:0/4117093468 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000fb50 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac8a7fc640 1 -- 172.21.15.62:0/4117093468 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9000fcf0 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac94161fe0 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac8a7fc640 1 -- 172.21.15.62:0/4117093468 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000fea0 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.241+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941625a0 con 0x7fac80004800 2024-04-26T22:28:46.244 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.243+0000 7fac8a7fc640 1 -- 172.21.15.62:0/4117093468 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fac90012040 con 0x7fac80004800 2024-04-26T22:28:46.249 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.243+0000 7fac8a7fc640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 0x7fac7806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.243+0000 7fac8a7fc640 1 -- 172.21.15.62:0/4117093468 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fac9008c240 con 0x7fac80004800 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.243+0000 7fac9927c640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 0x7fac7806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.244+0000 7fac9e9ff740 1 Processor -- start 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.244+0000 7fac9927c640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 0x7fac7806ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac84033000 tx=0x7fac8402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.244+0000 7fac9e9ff740 1 -- start start 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.244+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac0548e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 0x55ac054ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac053c8740 con 0x55ac05464120 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac053643a0 con 0x55ac054abce0 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55ac053c4700 con 0x55ac0548e440 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.250 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38180/0 (socket says 172.21.15.62:38180) 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 -- 172.21.15.62:0/2916549577 learned_addr learned my addr 172.21.15.62:0/2916549577 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9927c640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 0x55ac054ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac99a7d640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac0548e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 -- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 msgr2=0x55ac0548e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac0548e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 -- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 msgr2=0x55ac054ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 0x55ac054ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 -- 172.21.15.62:0/2916549577 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac05380750 con 0x55ac05464120 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9927c640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 0x55ac054ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac9a27e640 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fac80004780 tx=0x7fac8c001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7ac1dbd6b31144 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac777fe640 1 -- 172.21.15.62:0/2916549577 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c005400 con 0x55ac05464120 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac777fe640 1 -- 172.21.15.62:0/2916549577 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8c01a070 con 0x55ac05464120 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.245+0000 7fac777fe640 1 -- 172.21.15.62:0/2916549577 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0057a0 con 0x55ac05464120 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2916549577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 msgr2=0x55ac054aa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.251 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fac80004780 tx=0x7fac8c001610 comp rx=0 tx=0).stop 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2916549577 shutdown_connections 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac05464120 0x55ac054aa3a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054abce0 0x55ac054ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2916549577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac0548e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2916549577 >> 172.21.15.62:0/2916549577 conn(0x55ac05460a50 msgr2=0x55ac054a9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2916549577 shutdown_connections 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.246+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2916549577 wait complete. 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.247+0000 7fac9e9ff740 1 -- start start 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.247+0000 7fac9e9ff740 1 Processor -- start 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.247+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac0548e440 0x55ac054b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.247+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac054a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.247+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac05464ec0 con 0x55ac054a1420 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac0534ffe0 con 0x55ac0548e440 2024-04-26T22:28:46.252 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55ac0545e980 con 0x55ac054abce0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9e9ff740 10 client.0 _ll_register_callbacks cb 0x55ac05366150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38184/0 (socket says 172.21.15.62:38184) 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 -- 172.21.15.62:0/1821895587 learned_addr learned my addr 172.21.15.62:0/1821895587 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9927c640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac054a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac99a7d640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac0548e440 0x55ac054b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 -- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 msgr2=0x55ac054a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac054a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 -- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac0548e440 msgr2=0x55ac054b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac0548e440 0x55ac054b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 -- 172.21.15.62:0/1821895587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac050da2d0 con 0x55ac054a1420 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9a27e640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fac8c017040 tx=0x7fac8c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c5c87e7ec663c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c023070 con 0x55ac054a1420 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fac8c015c30 con 0x55ac054a1420 2024-04-26T22:28:46.253 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.248+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac8c01a070 con 0x55ac054a1420 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fac8c030050 con 0x55ac054a1420 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 10 client.5712 fetch_fsmap learned FSMap version 23 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55ac051a46b0 con 0x55ac054a1420 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8c01a070 con 0x55ac054a1420 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fac8c033020 con 0x55ac054a1420 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 10 client.5712 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 10 client.5712 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 20 client.5712 populate_metadata read hostname 'smithi062' 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05490370 0x55ac05490750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 4 client.5712 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T22:28:46.254 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 --> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac053d6350 con 0x55ac05490370 2024-04-26T22:28:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac99a7d640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05490370 0x55ac05490750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac74ff9640 10 client.5712 ms_handle_connect on v2:172.21.15.174:6834/4128033051 2024-04-26T22:28:46.255 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.249+0000 7fac99a7d640 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05490370 0x55ac05490750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af922a788bfeb19d server_cookie=869eeed13e28510b in_seq=0 out_seq=0 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac74ff9640 1 -- 172.21.15.62:0/1821895587 <== mds.0 v2:172.21.15.174:6834/4128033051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac053d6350 con 0x55ac05490370 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac74ff9640 10 client.5712 handle_command_reply: tid=0 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 client.5712 shutdown 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 msgr2=0x55ac054aa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fac8c017040 tx=0x7fac8c001510 comp rx=0 tx=0).stop 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 shutdown_connections 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05490370 0x55ac05490750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054a1420 0x55ac054aa3f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac054a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.256 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac74ff9640 0 client.5712 ms_handle_reset on v2:172.21.15.174:6834/4128033051 2024-04-26T22:28:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/1821895587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac0548e440 0x55ac054b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.250+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 >> 172.21.15.62:0/1821895587 conn(0x55ac05464120 msgr2=0x55ac054ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.251+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 shutdown_connections 2024-04-26T22:28:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.251+0000 7fac9e9ff740 1 -- 172.21.15.62:0/1821895587 wait complete. 2024-04-26T22:28:46.257 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.251+0000 7fac9e9ff740 20 client.5712 trim_cache size 0 max 16384 2024-04-26T22:28:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.269+0000 7fac9e9ff740 1 Processor -- start 2024-04-26T22:28:46.269 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.269+0000 7fac9e9ff740 1 -- start start 2024-04-26T22:28:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.269+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac05461ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.269+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac054ab390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.269+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.271 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fac94200eb0 con 0x55ac054abce0 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac942002a0 con 0x55ac054a1420 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fac94200860 con 0x55ac0548e440 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38188/0 (socket says 172.21.15.62:38188) 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 -- 172.21.15.62:0/3145614666 learned_addr learned my addr 172.21.15.62:0/3145614666 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 -- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 msgr2=0x55ac05461ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac05461ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 -- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 msgr2=0x55ac054ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac054ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 -- 172.21.15.62:0/3145614666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0545ee50 con 0x55ac054abce0 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac9a27e640 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fac8c003c40 tx=0x7fac8c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ced5b8d328dc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac76ffd640 1 -- 172.21.15.62:0/3145614666 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c023070 con 0x55ac054abce0 2024-04-26T22:28:46.272 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac76ffd640 1 -- 172.21.15.62:0/3145614666 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8c039070 con 0x55ac054abce0 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.270+0000 7fac76ffd640 1 -- 172.21.15.62:0/3145614666 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c02d070 con 0x55ac054abce0 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- 172.21.15.62:0/3145614666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 msgr2=0x55ac054cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fac8c003c40 tx=0x7fac8c020000 comp rx=0 tx=0).stop 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- 172.21.15.62:0/3145614666 shutdown_connections 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac054abce0 0x55ac054cc210 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac054ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/3145614666 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac0548e440 0x55ac05461ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- 172.21.15.62:0/3145614666 >> 172.21.15.62:0/3145614666 conn(0x55ac05464120 msgr2=0x55ac05463210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- 172.21.15.62:0/3145614666 shutdown_connections 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- 172.21.15.62:0/3145614666 wait complete. 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 -- start start 2024-04-26T22:28:46.273 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.271+0000 7fac9e9ff740 1 Processor -- start 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac05502430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac055157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ac054aae00 con 0x55ac0548e440 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac05464ec0 con 0x55ac054a1420 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55ac054bd040 con 0x55ac054abce0 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac9e9ff740 10 client.0 _ll_register_callbacks cb 0x7fac8c003e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T22:28:46.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac99a7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac99a7d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38202/0 (socket says 172.21.15.62:38202) 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.272+0000 7fac99a7d640 1 -- 172.21.15.62:0/2393728476 learned_addr learned my addr 172.21.15.62:0/2393728476 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9a27e640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac055157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9927c640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac05502430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 -- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 msgr2=0x55ac055157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac055157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 -- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 msgr2=0x55ac05502430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac05502430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 -- 172.21.15.62:0/2393728476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0548cc80 con 0x55ac0548e440 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9a27e640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac055157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9927c640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac05502430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac99a7d640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac90004790 tx=0x7fac9005a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fd9619411fd8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90085040 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mon.0 v2:172.21.15.62:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fac90056680 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac8c039070 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fac90080050 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9e9ff740 10 client.5718 fetch_fsmap learned FSMap version 23 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55ac055069f0 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mon.0 v2:172.21.15.62:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8c039070 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.273+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mon.0 v2:172.21.15.62:3300/0 5 ==== fsmap(e 23) ==== 3494+0+0 (secure 0 0 0) 0x7fac90083020 con 0x55ac0548e440 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 10 client.5718 fetch_fsmap finished waiting for FSMap version 23 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 10 client.5718 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 20 client.5718 populate_metadata read hostname 'smithi062' 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05462660 0x55ac054889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 4 client.5718 mds_command: new command op to 5559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 --> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] -- command(tid 0: {"prefix": "status"}) -- 0x55ac053d01c0 con 0x55ac05462660 2024-04-26T22:28:46.279 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9927c640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05462660 0x55ac054889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9927c640 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05462660 0x55ac054889b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229bb6e22f992a20 server_cookie=29d46820c8af6919 in_seq=0 out_seq=0 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac74ff9640 10 client.5718 ms_handle_connect on v2:172.21.15.174:6834/4128033051 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac74ff9640 1 -- 172.21.15.62:0/2393728476 <== mds.0 v2:172.21.15.174:6834/4128033051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55ac053d01c0 con 0x55ac05462660 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac74ff9640 10 client.5718 handle_command_reply: tid=0 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.274+0000 7fac9e9ff740 1 client.5718 shutdown 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 msgr2=0x55ac0548c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fac90004790 tx=0x7fac9005a0d0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 shutdown_connections 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:6834/4128033051,v1:172.21.15.174:6836/4128033051] conn(0x55ac05462660 0x55ac054889b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac054abce0 0x55ac055157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac054a1420 0x55ac05502430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac74ff9640 0 client.5718 ms_handle_reset on v2:172.21.15.174:6834/4128033051 2024-04-26T22:28:46.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 --2- 172.21.15.62:0/2393728476 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ac0548e440 0x55ac0548c450 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 >> 172.21.15.62:0/2393728476 conn(0x55ac05464120 msgr2=0x55ac054cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 shutdown_connections 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 1 -- 172.21.15.62:0/2393728476 wait complete. 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9e9ff740 20 client.5718 trim_cache size 0 max 16384 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "cluster_fsid": "f70b0a7c-519a-45f1-aa13-a1f2de1080a4", 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "whoami": 0, 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "id": 5559, 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "want_state": "up:active", 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "state": "up:active", 2024-04-26T22:28:46.281 INFO:teuthology.orchestra.run.smithi062.stdout: "fs_name": "cephfs", 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout: "rank_uptime": 9.132503625, 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout: "mdsmap_epoch": 23, 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch": 43, 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_epoch_barrier": 43, 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout: "uptime": 14.460939134 2024-04-26T22:28:46.282 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 msgr2=0x7fac7806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 0x7fac7806ee40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac84033000 tx=0x7fac8402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 msgr2=0x7fac9415a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fac90011a00 tx=0x7fac90011ec0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 shutdown_connections 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fac7806c980 0x7fac7806ee40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac9415ac70 0x7fac9415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac940caa60 0x7fac940caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 --2- 172.21.15.62:0/4117093468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fac80004800 0x7fac9415a530 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.283 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 >> 172.21.15.62:0/4117093468 conn(0x7fac9409ba50 msgr2=0x7fac9409be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 shutdown_connections 2024-04-26T22:28:46.284 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.275+0000 7fac9bce1640 1 -- 172.21.15.62:0/4117093468 wait complete. 2024-04-26T22:28:46.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:28:46.386 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.386+0000 7f0928264640 1 Processor -- start 2024-04-26T22:28:46.387 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 -- start start 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f09200d22e0 0x7f09200b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f092015f760 0x7f09200b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09200d3de0 con 0x7f092015f380 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f09200d43a0 con 0x7f092015f760 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0928264640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f09200d40c0 con 0x7f09200d22e0 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.387+0000 7f0926000640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f092015f380 0x7f09200d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49858/0 (socket says 172.21.15.62:49858) 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926000640 1 -- 172.21.15.62:0/365426225 learned_addr learned my addr 172.21.15.62:0/365426225 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:46.388 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926801640 1 --2- 172.21.15.62:0/365426225 >> v2:172.21.15.62:3300/0 conn(0x7f092015f760 0x7f09200b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f09257ff640 1 --2- 172.21.15.62:0/365426225 >> v2:172.21.15.174:3300/0 conn(0x7f09200d22e0 0x7f09200b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0924ffe640 1 -- 172.21.15.62:0/365426225 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847231120 0 0) 0x7f09200d3de0 con 0x7f092015f380 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0924ffe640 1 -- 172.21.15.62:0/365426225 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f090c003520 con 0x7f092015f380 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926801640 1 -- 172.21.15.62:0/365426225 >> v2:172.21.15.174:3300/0 conn(0x7f09200d22e0 msgr2=0x7f09200b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926801640 1 --2- 172.21.15.62:0/365426225 >> v2:172.21.15.174:3300/0 conn(0x7f09200d22e0 0x7f09200b8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926801640 1 -- 172.21.15.62:0/365426225 >> v1:172.21.15.62:6789/0 conn(0x7f092015f380 legacy=0x7f09200d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.389 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.388+0000 7f0926801640 1 -- 172.21.15.62:0/365426225 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09200b8280 con 0x7f092015f760 2024-04-26T22:28:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0926801640 1 --2- 172.21.15.62:0/365426225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200b8fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09180097a0 tx=0x7f091802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1121904920a558 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0924ffe640 1 -- 172.21.15.62:0/365426225 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918037070 con 0x7f092015f760 2024-04-26T22:28:46.390 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0924ffe640 1 -- 172.21.15.62:0/365426225 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0918032440 con 0x7f092015f760 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0924ffe640 1 -- 172.21.15.62:0/365426225 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180327f0 con 0x7f092015f760 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0928264640 1 -- 172.21.15.62:0/365426225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 msgr2=0x7f09200b8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.389+0000 7f0928264640 1 --2- 172.21.15.62:0/365426225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200b8fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f09180097a0 tx=0x7f091802a270 comp rx=0 tx=0).stop 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 -- 172.21.15.62:0/365426225 shutdown_connections 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 --2- 172.21.15.62:0/365426225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200b8fc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 --2- 172.21.15.62:0/365426225 >> v2:172.21.15.174:3300/0 conn(0x7f09200d22e0 0x7f09200b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 -- 172.21.15.62:0/365426225 >> 172.21.15.62:0/365426225 conn(0x7f092009e560 msgr2=0x7f09200d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 -- 172.21.15.62:0/365426225 shutdown_connections 2024-04-26T22:28:46.391 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.390+0000 7f0928264640 1 -- 172.21.15.62:0/365426225 wait complete. 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.391+0000 7f0928264640 1 Processor -- start 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.391+0000 7f0928264640 1 -- start start 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.391+0000 7f0928264640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09200d22e0 0x7f092015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092015f380 0x7f09200c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.393 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09200d1280 con 0x7f092015f760 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09200d1100 con 0x7f092015f380 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f09200d5b50 con 0x7f09200d22e0 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38220/0 (socket says 172.21.15.62:38220) 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 -- 172.21.15.62:0/1935237535 learned_addr learned my addr 172.21.15.62:0/1935237535 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926000640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09200d22e0 0x7f092015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f09257ff640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092015f380 0x7f09200c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 -- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09200d22e0 msgr2=0x7f092015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09200d22e0 0x7f092015b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 -- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092015f380 msgr2=0x7f09200c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092015f380 0x7f09200c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 -- 172.21.15.62:0/1935237535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09200ccad0 con 0x7f092015f760 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0926801640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f091802a2b0 tx=0x7f09180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.394 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0918037040 con 0x7f092015f760 2024-04-26T22:28:46.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0918042070 con 0x7f092015f760 2024-04-26T22:28:46.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09200ccdb0 con 0x7f092015f760 2024-04-26T22:28:46.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09180044e0 con 0x7f092015f760 2024-04-26T22:28:46.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.392+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09200cd370 con 0x7f092015f760 2024-04-26T22:28:46.395 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.393+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092006a360 con 0x7f092015f760 2024-04-26T22:28:46.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.394+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f0918032830 con 0x7f092015f760 2024-04-26T22:28:46.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.394+0000 7f09167fc640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 0x7f090406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:46.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.394+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f09180ac420 con 0x7f092015f760 2024-04-26T22:28:46.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.395+0000 7f0926000640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 0x7f090406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:46.396 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.395+0000 7f0926000640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 0x7f090406ee40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f091c005c00 tx=0x7f091c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:46.398 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.398+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f091807a860 con 0x7f092015f760 2024-04-26T22:28:46.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.624+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f09200cf5a0 con 0x7f092015f760 2024-04-26T22:28:46.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.624+0000 7f09167fc640 1 -- 172.21.15.62:0/1935237535 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f091807a160 con 0x7f092015f760 2024-04-26T22:28:46.624 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:28:46.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 msgr2=0x7f090406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 0x7f090406ee40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f091c005c00 tx=0x7f091c009040 comp rx=0 tx=0).stop 2024-04-26T22:28:46.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 msgr2=0x7f09200cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:46.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f091802a2b0 tx=0x7f09180046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 shutdown_connections 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f090406c980 0x7f090406ee40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f092015f760 0x7f09200cc360 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092015f380 0x7f09200c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 --2- 172.21.15.62:0/1935237535 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09200d22e0 0x7f092015b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 >> 172.21.15.62:0/1935237535 conn(0x7f092009e560 msgr2=0x7f09200cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 shutdown_connections 2024-04-26T22:28:46.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46.626+0000 7f0928264640 1 -- 172.21.15.62:0/1935237535 wait complete. 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi062.front.sepia.ceph.com 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T22:28:46.637 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T22:28:46.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:28:46.638 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:46.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:46.638 DEBUG:teuthology.orchestra.run.smithi062:> ip addr 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 ::1/128 scope host 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:14 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f0 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:a9:aa:15 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:46.654 INFO:teuthology.orchestra.run.smithi062.stdout: altname enp7s0f1 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c2 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 0c:c4:7a:bb:d1:c3 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: inet 172.21.15.62/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft 42667sec preferred_lft 42667sec 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::6c94:f27c:3545:34e3/64 scope link noprefixroute 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 46:95:32:b9:0b:f0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::2a:a6ff:fef2:1980/64 scope link 2024-04-26T22:28:46.655 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether d2:58:c7:f5:1b:26 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::d058:c7ff:fef5:1b26/64 scope link 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: link/ether 46:95:32:b9:0b:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: inet6 fe80::4495:32ff:feb9:bf0/64 scope link 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run.smithi062.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:46.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:46.656 DEBUG:teuthology.orchestra.run.smithi062:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:28:46.717 INFO:teuthology.orchestra.run.smithi062.stdout:1 2024-04-26T22:28:46.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:46.718 DEBUG:teuthology.orchestra.run.smithi062:> route 2024-04-26T22:28:46.775 INFO:teuthology.orchestra.run.smithi062.stdout:Kernel IP routing table 2024-04-26T22:28:46.776 INFO:teuthology.orchestra.run.smithi062.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:28:46.776 INFO:teuthology.orchestra.run.smithi062.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:28:46.776 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:28:46.776 INFO:teuthology.orchestra.run.smithi062.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> set -e 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:28:46.778 DEBUG:teuthology.orchestra.run.smithi062:> ') 2024-04-26T22:28:46.865 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:28:46.895 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:46.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:46.902 DEBUG:teuthology.orchestra.run.smithi062:> ip netns list 2024-04-26T22:28:46.917 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T22:28:46.917 INFO:teuthology.orchestra.run.smithi062.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T22:28:46.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:46.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:46.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo ip link set brx.0 up 2024-04-26T22:28:46.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:46.982 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:47.037 INFO:teuthology.orchestra.run.smithi062.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T22:28:47.037 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:47.038 DEBUG:teuthology.orchestra.run.smithi062:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:47.092 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:28:47.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:47.093 DEBUG:teuthology.orchestra.run.smithi062:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:47.175 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:47.175 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:47.175 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:47.185 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:47.185 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:47.185 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:47.185 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:47.185 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:invalid new device string format 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:28:47.190 INFO:teuthology.orchestra.run.smithi062.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:28:47.191 INFO:teuthology.orchestra.run.smithi062.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T22:28:47.191 INFO:teuthology.orchestra.run.smithi062.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:28:47.191 INFO:teuthology.orchestra.run.smithi062.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:28:47.191 INFO:teuthology.orchestra.run.smithi062.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:28:47.191 INFO:teuthology.orchestra.run.smithi062.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:28:47.192 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:28:47.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:47.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T22:28:47.258 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:28:47.350 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:28:47.356 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:47.362 INFO:teuthology.orchestra.run.smithi062.stdout:client5365 2024-04-26T22:28:47.362 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-26T22:28:47.362 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:47.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T22:28:47.518 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.518+0000 7f74c0d9d640 1 Processor -- start 2024-04-26T22:28:47.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.518+0000 7f74c0d9d640 1 -- start start 2024-04-26T22:28:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.519+0000 7f74c0d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74bc162b40 0x7f74bc0d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.519+0000 7f74c0d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 0x7f74bc0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.519+0000 7f74c0d9d640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc0d0210 con 0x7f74bc0d1850 2024-04-26T22:28:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.519+0000 7f74c0d9d640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc0d41c0 con 0x7f74bc162b40 2024-04-26T22:28:47.520 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.519+0000 7f74c0d9d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0d04f0 con 0x7f74bc0d3820 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74ba59c640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f74bc162b40 0x7f74bc0d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74bad9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f74bc0d1850 0x7f74bc0d1ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49874/0 (socket says 172.21.15.62:49874) 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74bad9d640 1 -- 172.21.15.62:0/2838045663 learned_addr learned my addr 172.21.15.62:0/2838045663 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74b9d9b640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 0x7f74bc0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74ba59c640 1 -- 172.21.15.62:0/2838045663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 msgr2=0x7f74bc0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74ba59c640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 0x7f74bc0d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74ba59c640 1 -- 172.21.15.62:0/2838045663 >> v1:172.21.15.62:6789/0 conn(0x7f74bc0d1850 legacy=0x7f74bc0d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74b959a640 1 -- 172.21.15.62:0/2838045663 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600043994 0 0) 0x7f74bc0d0210 con 0x7f74bc0d1850 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74ba59c640 1 -- 172.21.15.62:0/2838045663 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0d07d0 con 0x7f74bc162b40 2024-04-26T22:28:47.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.520+0000 7f74b9d9b640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 0x7f74bc0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74ba59c640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc0d5bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f74b40097a0 tx=0x7f74b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740aa886ff538b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74b959a640 1 -- 172.21.15.62:0/2838045663 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4037070 con 0x7f74bc162b40 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74b959a640 1 -- 172.21.15.62:0/2838045663 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b4032440 con 0x7f74bc162b40 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74b959a640 1 -- 172.21.15.62:0/2838045663 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4032810 con 0x7f74bc162b40 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 -- 172.21.15.62:0/2838045663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 msgr2=0x7f74bc0d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc0d5bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f74b40097a0 tx=0x7f74b402a210 comp rx=0 tx=0).stop 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 -- 172.21.15.62:0/2838045663 shutdown_connections 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d3820 0x7f74bc0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/2838045663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc0d5bd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 -- 172.21.15.62:0/2838045663 >> 172.21.15.62:0/2838045663 conn(0x7f74bc09e550 msgr2=0x7f74bc0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 -- 172.21.15.62:0/2838045663 shutdown_connections 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.521+0000 7f74c0d9d640 1 -- 172.21.15.62:0/2838045663 wait complete. 2024-04-26T22:28:47.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 Processor -- start 2024-04-26T22:28:47.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 -- start start 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d1850 0x7f74bc15e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74bc0d3820 0x7f74bc161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0b5bb0 con 0x7f74bc162b40 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0d0d90 con 0x7f74bc0d3820 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.523+0000 7f74c0d9d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0d6110 con 0x7f74bc0d1850 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38240/0 (socket says 172.21.15.62:38240) 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 -- 172.21.15.62:0/3233351315 learned_addr learned my addr 172.21.15.62:0/3233351315 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74ba59c640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d1850 0x7f74bc15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74b9d9b640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74bc0d3820 0x7f74bc161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 -- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d1850 msgr2=0x7f74bc15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d1850 0x7f74bc15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 -- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74bc0d3820 msgr2=0x7f74bc161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74bc0d3820 0x7f74bc161240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 -- 172.21.15.62:0/3233351315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc1f5180 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74bad9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c7b0 tx=0x7f74ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac018070 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74ac00a640 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc15ea80 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00aa30 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.524+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1f56d0 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.525+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc06a360 con 0x7f74bc162b40 2024-04-26T22:28:47.527 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.526+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f74ac01b070 con 0x7f74bc162b40 2024-04-26T22:28:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.526+0000 7f74b2ffd640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 0x7f749c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.526+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f74ac08b1a0 con 0x7f74bc162b40 2024-04-26T22:28:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.527+0000 7f74ba59c640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 0x7f749c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:47.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.527+0000 7f74ba59c640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 0x7f749c06ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74b4034000 tx=0x7f74b402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:47.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.531+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74ac0595e0 con 0x7f74bc162b40 2024-04-26T22:28:47.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.754+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74bc1622f0 con 0x7f74bc162b40 2024-04-26T22:28:47.755 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.755+0000 7f74b2ffd640 1 -- 172.21.15.62:0/3233351315 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f74ac058ee0 con 0x7f74bc162b40 2024-04-26T22:28:47.755 INFO:teuthology.orchestra.run.smithi062.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.756+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 msgr2=0x7f749c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.756+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 0x7f749c06ee40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74b4034000 tx=0x7f74b402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.756+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 msgr2=0x7f74bc161780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.756+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c7b0 tx=0x7f74ac00cc70 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 shutdown_connections 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f749c06c980 0x7f749c06ee40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74bc162b40 0x7f74bc161780 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74bc0d3820 0x7f74bc161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 --2- 172.21.15.62:0/3233351315 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f74bc0d1850 0x7f74bc15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 >> 172.21.15.62:0/3233351315 conn(0x7f74bc09e550 msgr2=0x7f74bc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:47.757 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 shutdown_connections 2024-04-26T22:28:47.758 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:47.757+0000 7f74c0d9d640 1 -- 172.21.15.62:0/3233351315 wait complete. 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T22:28:47.768 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:47.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:47.769 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:47.788 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f0 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f1 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42651sec preferred_lft 42651sec 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::daa0:5f76:a60a:7388/64 scope link noprefixroute 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether da:52:8b:02:eb:e8 brd ff:ff:ff:ff:ff:ff 2024-04-26T22:28:47.789 INFO:teuthology.orchestra.run.smithi174.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::d03c:ebff:fec0:f023/64 scope link 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether ea:9b:af:31:81:76 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::e89b:afff:fe31:8176/64 scope link 2024-04-26T22:28:47.790 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.791 INFO:teuthology.orchestra.run.smithi174.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T22:28:47.791 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether da:52:8b:02:eb:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T22:28:47.791 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::d852:8bff:fe02:ebe8/64 scope link 2024-04-26T22:28:47.791 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-04-26T22:28:47.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:47.791 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T22:28:47.855 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-04-26T22:28:47.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:47.857 DEBUG:teuthology.orchestra.run.smithi174:> route 2024-04-26T22:28:47.914 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2024-04-26T22:28:47.914 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T22:28:47.914 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T22:28:47.914 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T22:28:47.914 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T22:28:47.915 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-04-26T22:28:48.006 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T22:28:48.037 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:48.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:48.043 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-04-26T22:28:48.059 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T22:28:48.059 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T22:28:48.059 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:48.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:48.059 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.0 up 2024-04-26T22:28:48.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:48.123 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:48.178 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T22:28:48.178 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T22:28:48.178 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:48.232 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T22:28:48.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:48.233 DEBUG:teuthology.orchestra.run.smithi174:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:48.313 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:48.313 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:48.313 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:48.321 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:48.321 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:48.321 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:48.322 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:48.322 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:invalid new device string format 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:3300,172.21.15.174:3300,172.21.15.174:3301" 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: resolved to: "172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790" 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.62:6789,172.21.15.174:6789,172.21.15.174:6790:/ 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=f70b0a7c-519a-45f1-aa13-a1f2de1080a4" will pass to kernel 2024-04-26T22:28:48.327 INFO:teuthology.orchestra.run.smithi174.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T22:28:48.328 INFO:teuthology.orchestra.run.smithi174.stdout: You just mounted a file system that supports labels which does not 2024-04-26T22:28:48.328 INFO:teuthology.orchestra.run.smithi174.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T22:28:48.328 INFO:teuthology.orchestra.run.smithi174.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T22:28:48.328 INFO:teuthology.orchestra.run.smithi174.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T22:28:48.329 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T22:28:48.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:48.329 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T22:28:48.395 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T22:28:48.483 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T22:28:48.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T22:28:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T22:28:48.490 INFO:teuthology.orchestra.run.smithi174.stdout:client5739 2024-04-26T22:28:48.491 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T22:28:48.491 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T22:28:48.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-04-26T22:28:48.623 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.623+0000 7f39588c4640 1 Processor -- start 2024-04-26T22:28:48.624 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.623+0000 7f39588c4640 1 -- start start 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f39588c4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f395415f390 0x7f3954072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f39588c4640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39540b1e50 0x7f39540cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f39588c4640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f395415f770 con 0x7f395406e9c0 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f39588c4640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f39540d2db0 con 0x7f39540b1e50 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f39588c4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f39540d2ad0 con 0x7f395415f390 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f3951d9b640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f39540b1e50 0x7f39540cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f3952d9d640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f395406e9c0 0x7f39541631c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49888/0 (socket says 172.21.15.62:49888) 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f3952d9d640 1 -- 172.21.15.62:0/871946101 learned_addr learned my addr 172.21.15.62:0/871946101 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.624+0000 7f395259c640 1 --2- 172.21.15.62:0/871946101 >> v2:172.21.15.174:3300/0 conn(0x7f395415f390 0x7f3954072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f3951d9b640 1 -- 172.21.15.62:0/871946101 >> v2:172.21.15.174:3300/0 conn(0x7f395415f390 msgr2=0x7f3954072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f3951d9b640 1 --2- 172.21.15.62:0/871946101 >> v2:172.21.15.174:3300/0 conn(0x7f395415f390 0x7f3954072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f3951d9b640 1 -- 172.21.15.62:0/871946101 >> v1:172.21.15.62:6789/0 conn(0x7f395406e9c0 legacy=0x7f39541631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:48.625 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f395159a640 1 -- 172.21.15.62:0/871946101 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738808695 0 0) 0x7f395415f770 con 0x7f395406e9c0 2024-04-26T22:28:48.626 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f3951d9b640 1 -- 172.21.15.62:0/871946101 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39540d0c60 con 0x7f39540b1e50 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.625+0000 7f3951d9b640 1 --2- 172.21.15.62:0/871946101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39540b1e50 0x7f39540cd5b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3954070890 tx=0x7f393c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d221362dbc8f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f395159a640 1 -- 172.21.15.62:0/871946101 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c037070 con 0x7f39540b1e50 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f395159a640 1 -- 172.21.15.62:0/871946101 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f393c032440 con 0x7f39540b1e50 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f395159a640 1 -- 172.21.15.62:0/871946101 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c02ae60 con 0x7f39540b1e50 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 -- 172.21.15.62:0/871946101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39540b1e50 msgr2=0x7f39540cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 --2- 172.21.15.62:0/871946101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39540b1e50 0x7f39540cd5b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3954070890 tx=0x7f393c02a110 comp rx=0 tx=0).stop 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 -- 172.21.15.62:0/871946101 shutdown_connections 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 --2- 172.21.15.62:0/871946101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f39540b1e50 0x7f39540cd5b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 --2- 172.21.15.62:0/871946101 >> v2:172.21.15.174:3300/0 conn(0x7f395415f390 0x7f3954072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 -- 172.21.15.62:0/871946101 >> 172.21.15.62:0/871946101 conn(0x7f39540d2f60 msgr2=0x7f39540d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 -- 172.21.15.62:0/871946101 shutdown_connections 2024-04-26T22:28:48.627 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.626+0000 7f39588c4640 1 -- 172.21.15.62:0/871946101 wait complete. 2024-04-26T22:28:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 Processor -- start 2024-04-26T22:28:48.628 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 -- start start 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39540b1e50 0x7f39540ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f395415f390 0x7f39540ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f39540d1220 con 0x7f395406e9c0 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39540d2780 con 0x7f39540b1e50 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.628+0000 7f39588c4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f395406edf0 con 0x7f395415f390 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38264/0 (socket says 172.21.15.62:38264) 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 -- 172.21.15.62:0/2826071347 learned_addr learned my addr 172.21.15.62:0/2826071347 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f3952d9d640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f395415f390 0x7f39540ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f3951d9b640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39540b1e50 0x7f39540ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 -- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f395415f390 msgr2=0x7f39540ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:48.630 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f395415f390 0x7f39540ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 -- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39540b1e50 msgr2=0x7f39540ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39540b1e50 0x7f39540ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 -- 172.21.15.62:0/2826071347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39541e96a0 con 0x7f395406e9c0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f395259c640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e870 tx=0x7f394c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c017070 con 0x7f395406e9c0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394c00c940 con 0x7f395406e9c0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c00cd30 con 0x7f395406e9c0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3954146d80 con 0x7f395406e9c0 2024-04-26T22:28:48.631 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.629+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3954147340 con 0x7f395406e9c0 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.630+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39541ecce0 con 0x7f395406e9c0 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.631+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f394c01a070 con 0x7f395406e9c0 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.631+0000 7f394affd640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 0x7f393406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.631+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f394c08bd60 con 0x7f395406e9c0 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.632+0000 7f3951d9b640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 0x7f393406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:48.632 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.632+0000 7f3951d9b640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 0x7f393406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f393c034000 tx=0x7f393c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:48.635 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.636+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f394c059b50 con 0x7f395406e9c0 2024-04-26T22:28:48.862 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:48.861+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3954147720 con 0x7f395406e9c0 2024-04-26T22:28:49.372 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.372+0000 7f394affd640 1 -- 172.21.15.62:0/2826071347 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v24) ==== 136+0+0 (secure 0 0 0) 0x7f394c05de50 con 0x7f395406e9c0 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.374+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 msgr2=0x7f393406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.374+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 0x7f393406ee40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f393c034000 tx=0x7f393c02e000 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.374+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 msgr2=0x7f39540cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.374+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e870 tx=0x7f394c00ed30 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 shutdown_connections 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f393406c980 0x7f393406ee40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f395415f390 0x7f39540ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39540b1e50 0x7f39540ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 --2- 172.21.15.62:0/2826071347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f395406e9c0 0x7f39540cec70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 >> 172.21.15.62:0/2826071347 conn(0x7f39540d2f60 msgr2=0x7f39540ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:49.375 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 shutdown_connections 2024-04-26T22:28:49.376 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.375+0000 7f39588c4640 1 -- 172.21.15.62:0/2826071347 wait complete. 2024-04-26T22:28:49.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T22:28:49.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.495+0000 7ff604fc1640 1 Processor -- start 2024-04-26T22:28:49.496 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.495+0000 7ff604fc1640 1 -- start start 2024-04-26T22:28:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.496+0000 7ff604fc1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff6000d3820 0x7ff6000d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.496+0000 7ff604fc1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.496+0000 7ff604fc1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6000d0210 con 0x7ff600162b40 2024-04-26T22:28:49.497 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.496+0000 7ff604fc1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff6000d41c0 con 0x7ff6000d3820 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.496+0000 7ff604fc1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff6000d04f0 con 0x7ff6000b4ba0 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fed9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fdd9b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff6000d3820 0x7ff6000d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fe59c640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7ff600162b40 0x7ff6000d5bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.62:40880/0 (socket says 172.21.15.62:40880) 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fe59c640 1 -- 172.21.15.62:0/1827358568 learned_addr learned my addr 172.21.15.62:0/1827358568 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 msgr2=0x7ff6000b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fdd9b640 1 --2- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/1827358568 >> v1:172.21.15.174:6789/0 conn(0x7ff600162b40 legacy=0x7ff6000d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.498 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.497+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/1827358568 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000d07d0 con 0x7ff6000d3820 2024-04-26T22:28:49.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.498+0000 7ff5fdd9b640 1 --2- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6000d3820 0x7ff6000d3c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80097a0 tx=0x7ff5e802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c40cb4c32c50360 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.499 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.498+0000 7ff5fd59a640 1 -- 172.21.15.62:0/1827358568 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8037070 con 0x7ff6000d3820 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.498+0000 7ff5fd59a640 1 -- 172.21.15.62:0/1827358568 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e8032440 con 0x7ff6000d3820 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.498+0000 7ff5fd59a640 1 -- 172.21.15.62:0/1827358568 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8032810 con 0x7ff6000d3820 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 -- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6000d3820 msgr2=0x7ff6000d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 --2- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6000d3820 0x7ff6000d3c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80097a0 tx=0x7ff5e802a180 comp rx=0 tx=0).stop 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 -- 172.21.15.62:0/1827358568 shutdown_connections 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 --2- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 --2- 172.21.15.62:0/1827358568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6000d3820 0x7ff6000d3c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 -- 172.21.15.62:0/1827358568 >> 172.21.15.62:0/1827358568 conn(0x7ff60009e550 msgr2=0x7ff6000d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 -- 172.21.15.62:0/1827358568 shutdown_connections 2024-04-26T22:28:49.500 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.499+0000 7ff604fc1640 1 -- 172.21.15.62:0/1827358568 wait complete. 2024-04-26T22:28:49.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.500+0000 7ff604fc1640 1 Processor -- start 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.500+0000 7ff604fc1640 1 -- start start 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000baa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600162b40 0x7ff6000b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000d12b0 con 0x7ff6000d3820 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000d0d90 con 0x7ff600162b40 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff604fc1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff6000cfe30 con 0x7ff6000b4ba0 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38278/0 (socket says 172.21.15.62:38278) 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/2208222841 learned_addr learned my addr 172.21.15.62:0/2208222841 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 msgr2=0x7ff6000baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.503 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fed9d640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600162b40 0x7ff6000b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600162b40 msgr2=0x7ff6000b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600162b40 0x7ff6000b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 -- 172.21.15.62:0/2208222841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000cc5f0 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.501+0000 7ff5fdd9b640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8035040 tx=0x7ff5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.502+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8037040 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.502+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e8030500 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.502+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6000b9470 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.502+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8030910 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.502+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6000b9a30 con 0x7ff6000d3820 2024-04-26T22:28:49.504 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.503+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60006a360 con 0x7ff6000d3820 2024-04-26T22:28:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.503+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff5e8044020 con 0x7ff6000d3820 2024-04-26T22:28:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.503+0000 7ff5f6ffd640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 0x7ff5e006ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.504+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e80abe10 con 0x7ff6000d3820 2024-04-26T22:28:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.504+0000 7ff5fe59c640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 0x7ff5e006ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.505 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.504+0000 7ff5fe59c640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 0x7ff5e006ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8000f60 tx=0x7ff5f8007a80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.508 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.508+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff5e807a2e0 con 0x7ff6000d3820 2024-04-26T22:28:49.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.720+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff6000d2370 con 0x7ff5e006c980 2024-04-26T22:28:49.722 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.722+0000 7ff5f6ffd640 1 -- 172.21.15.62:0/2208222841 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff6000d2370 con 0x7ff5e006c980 2024-04-26T22:28:49.722 INFO:teuthology.orchestra.run.smithi062.stdout:[ 2024-04-26T22:28:49.722 INFO:teuthology.orchestra.run.smithi062.stdout: { 2024-04-26T22:28:49.723 INFO:teuthology.orchestra.run.smithi062.stdout: "name": "cephfs" 2024-04-26T22:28:49.723 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-26T22:28:49.723 INFO:teuthology.orchestra.run.smithi062.stdout:] 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 msgr2=0x7ff5e006ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 0x7ff5e006ee40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8000f60 tx=0x7ff5f8007a80 comp rx=0 tx=0).stop 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 msgr2=0x7ff6000b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8035040 tx=0x7ff5e80046c0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 shutdown_connections 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff5e006c980 0x7ff5e006ee40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff600162b40 0x7ff6000b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff6000d3820 0x7ff6000b87f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.725 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 --2- 172.21.15.62:0/2208222841 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff6000b4ba0 0x7ff6000baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 >> 172.21.15.62:0/2208222841 conn(0x7ff60009e550 msgr2=0x7ff60006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:49.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 shutdown_connections 2024-04-26T22:28:49.726 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.724+0000 7ff604fc1640 1 -- 172.21.15.62:0/2208222841 wait complete. 2024-04-26T22:28:49.735 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T22:28:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.839+0000 7f9601a08640 1 Processor -- start 2024-04-26T22:28:49.839 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.839+0000 7f9601a08640 1 -- start start 2024-04-26T22:28:49.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f9601a08640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95fc160800 0x7f95fc1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f9601a08640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f95fc0d5b60 0x7f95fc0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.845 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f9601a08640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc0d0930 con 0x7f95fc0d2150 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f9601a08640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc0d25c0 con 0x7f95fc160800 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f9601a08640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc0d0c00 con 0x7f95fc0d5b60 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95fc160800 0x7f95fc1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f95fc160800 0x7f95fc1610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38290/0 (socket says 172.21.15.62:38290) 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 -- 172.21.15.62:0/827049749 learned_addr learned my addr 172.21.15.62:0/827049749 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95fb7fe640 1 --1- 172.21.15.62:0/827049749 >> v1:172.21.15.62:6789/0 conn(0x7f95fc0d2150 0x7f95fc0d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49900/0 (socket says 172.21.15.62:49900) 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95fa7fc640 1 --2- 172.21.15.62:0/827049749 >> v2:172.21.15.174:3300/0 conn(0x7f95fc0d5b60 0x7f95fc0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 -- 172.21.15.62:0/827049749 >> v2:172.21.15.174:3300/0 conn(0x7f95fc0d5b60 msgr2=0x7f95fc0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 --2- 172.21.15.62:0/827049749 >> v2:172.21.15.174:3300/0 conn(0x7f95fc0d5b60 0x7f95fc0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 -- 172.21.15.62:0/827049749 >> v1:172.21.15.62:6789/0 conn(0x7f95fc0d2150 legacy=0x7f95fc0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.840+0000 7f95faffd640 1 -- 172.21.15.62:0/827049749 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0d0ee0 con 0x7f95fc160800 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.841+0000 7f95faffd640 1 --2- 172.21.15.62:0/827049749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc160800 0x7f95fc1610c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0b6090 tx=0x7f95f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9fdfb32bb2628 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.846 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.841+0000 7f95f9ffb640 1 -- 172.21.15.62:0/827049749 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4037070 con 0x7f95fc160800 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.841+0000 7f95f9ffb640 1 -- 172.21.15.62:0/827049749 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f4032440 con 0x7f95fc160800 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.841+0000 7f95f9ffb640 1 -- 172.21.15.62:0/827049749 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4032810 con 0x7f95fc160800 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 -- 172.21.15.62:0/827049749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc160800 msgr2=0x7f95fc1610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 --2- 172.21.15.62:0/827049749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc160800 0x7f95fc1610c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0b6090 tx=0x7f95f402a270 comp rx=0 tx=0).stop 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 -- 172.21.15.62:0/827049749 shutdown_connections 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 --2- 172.21.15.62:0/827049749 >> v2:172.21.15.174:3300/0 conn(0x7f95fc0d5b60 0x7f95fc0d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 --2- 172.21.15.62:0/827049749 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc160800 0x7f95fc1610c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 -- 172.21.15.62:0/827049749 >> 172.21.15.62:0/827049749 conn(0x7f95fc09e550 msgr2=0x7f95fc0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 -- 172.21.15.62:0/827049749 shutdown_connections 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.842+0000 7f9601a08640 1 -- 172.21.15.62:0/827049749 wait complete. 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.843+0000 7f9601a08640 1 Processor -- start 2024-04-26T22:28:49.847 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.843+0000 7f9601a08640 1 -- start start 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 0x7f95fc156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95fc160800 0x7f95fc1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0d30f0 con 0x7f95fc0d5b60 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0d2e10 con 0x7f95fc0d2150 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f9601a08640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0b6300 con 0x7f95fc160800 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38306/0 (socket says 172.21.15.62:38306) 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 -- 172.21.15.62:0/518825975 learned_addr learned my addr 172.21.15.62:0/518825975 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fb7fe640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95fc160800 0x7f95fc1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95faffd640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 0x7f95fc156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 -- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95fc160800 msgr2=0x7f95fc1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95fc160800 0x7f95fc1532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 -- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 msgr2=0x7f95fc156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:49.848 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 0x7f95fc156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 -- 172.21.15.62:0/518825975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc1565c0 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95faffd640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 0x7f95fc156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95fa7fc640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f95f000d800 tx=0x7f95f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.844+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f000b6c0 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.845+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f0019070 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.845+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc0b96b0 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.845+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f000ba80 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.845+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc0cca10 con 0x7f95fc0d5b60 2024-04-26T22:28:49.849 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.846+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc06a360 con 0x7f95fc0d5b60 2024-04-26T22:28:49.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.846+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f95f000bc20 con 0x7f95fc0d5b60 2024-04-26T22:28:49.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.846+0000 7f95e3fff640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 0x7f95e806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:49.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.847+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f95f008bfc0 con 0x7f95fc0d5b60 2024-04-26T22:28:49.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.847+0000 7f95faffd640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 0x7f95e806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:49.850 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.847+0000 7f95faffd640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 0x7f95e806ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f95f4034000 tx=0x7f95f402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:49.851 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:49.851+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f95fc0b96b0 con 0x7f95fc0d5b60 2024-04-26T22:28:50.038 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.038+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f95fc0d4530 con 0x7f95fc0d5b60 2024-04-26T22:28:50.039 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.038+0000 7f95e3fff640 1 -- 172.21.15.62:0/518825975 <== mon.0 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f95f001c360 con 0x7f95fc0d5b60 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 msgr2=0x7f95e806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 0x7f95e806ee70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f95f4034000 tx=0x7f95f402e000 comp rx=0 tx=0).stop 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 msgr2=0x7f95fc0ba270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f95f000d800 tx=0x7f95f000dcc0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 shutdown_connections 2024-04-26T22:28:50.041 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f95e806c9b0 0x7f95e806ee70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f95fc160800 0x7f95fc1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f95fc0d5b60 0x7f95fc0ba270 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 --2- 172.21.15.62:0/518825975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc0d2150 0x7f95fc156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 >> 172.21.15.62:0/518825975 conn(0x7f95fc09e550 msgr2=0x7f95fc161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 shutdown_connections 2024-04-26T22:28:50.042 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.041+0000 7f9601a08640 1 -- 172.21.15.62:0/518825975 wait complete. 2024-04-26T22:28:50.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp61490 2024-04-26T22:28:50.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.156+0000 7f6082719640 1 Processor -- start 2024-04-26T22:28:50.156 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.156+0000 7f6082719640 1 -- start start 2024-04-26T22:28:50.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 0x7f607c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 0x7f607c0d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.157 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f607c0d1fe0 0x7f607c162fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7f607c0d1190 con 0x7f607c0d1fe0 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f607c0d2ed0 con 0x7f607c0d0d30 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6082719640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0d31b0 con 0x7f607c15f470 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f607c0d1fe0 0x7f607c162fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7f607c0d1fe0 0x7f607c162fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38320/0 (socket says 172.21.15.62:38320) 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 -- 172.21.15.62:0/1086063699 learned_addr learned my addr 172.21.15.62:0/1086063699 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f607bfff640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 0x7f607c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f607b7fe640 1 --2- 172.21.15.62:0/1086063699 >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 0x7f607c0d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 -- 172.21.15.62:0/1086063699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 msgr2=0x7f607c0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 0x7f607c0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 -- 172.21.15.62:0/1086063699 >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 msgr2=0x7f607c0d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 --2- 172.21.15.62:0/1086063699 >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 0x7f607c0d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f6080cb6640 1 -- 172.21.15.62:0/1086063699 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c0d3490 con 0x7f607c0d1fe0 2024-04-26T22:28:50.158 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f607bfff640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 0x7f607c0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:50.159 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.157+0000 7f607b7fe640 1 --2- 172.21.15.62:0/1086063699 >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 0x7f607c0d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.158+0000 7f6080cb6640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c0d1fe0 0x7f607c162fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8240e824dd90194 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.158+0000 7f607affd640 1 -- 172.21.15.62:0/1086063699 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074036070 con 0x7f607c0d1fe0 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.158+0000 7f607affd640 1 -- 172.21.15.62:0/1086063699 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074031600 con 0x7f607c0d1fe0 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.158+0000 7f607affd640 1 -- 172.21.15.62:0/1086063699 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60740319d0 con 0x7f607c0d1fe0 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 -- 172.21.15.62:0/1086063699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c0d1fe0 msgr2=0x7f607c162fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c0d1fe0 0x7f607c162fd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402a6c0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 -- 172.21.15.62:0/1086063699 shutdown_connections 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c0d1fe0 0x7f607c162fd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.160 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 --2- 172.21.15.62:0/1086063699 >> v2:172.21.15.174:3300/0 conn(0x7f607c0d0d30 0x7f607c0d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 --2- 172.21.15.62:0/1086063699 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c15f470 0x7f607c0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 -- 172.21.15.62:0/1086063699 >> 172.21.15.62:0/1086063699 conn(0x7f607c09e510 msgr2=0x7f607c0d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 -- 172.21.15.62:0/1086063699 shutdown_connections 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.159+0000 7f6082719640 1 -- 172.21.15.62:0/1086063699 wait complete. 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.160+0000 7f6082719640 1 Processor -- start 2024-04-26T22:28:50.161 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.160+0000 7f6082719640 1 -- start start 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f607c0d0d30 0x7f607c0bab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 0x7f607c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f607c15f850 con 0x7f607c15f470 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0d3a50 con 0x7f607c0d0d30 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0d3f70 con 0x7f607c0d1fe0 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38322/0 (socket says 172.21.15.62:38322) 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 -- 172.21.15.62:0/4166995790 learned_addr learned my addr 172.21.15.62:0/4166995790 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:50.163 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f607bfff640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f607c0d0d30 0x7f607c0bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f607b7fe640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 0x7f607c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 -- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 msgr2=0x7f607c0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 0x7f607c0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 -- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f607c0d0d30 msgr2=0x7f607c0bab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f607c0d0d30 0x7f607c0bab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 -- 172.21.15.62:0/4166995790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c0cf230 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f607b7fe640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 0x7f607c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6080cb6640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6074009f70 tx=0x7f6074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074036050 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074042070 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c15e120 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.161+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074009510 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.162+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c0ce470 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.162+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c06a360 con 0x7f607c15f470 2024-04-26T22:28:50.164 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.163+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f6074007bb0 con 0x7f607c15f470 2024-04-26T22:28:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.163+0000 7f6078ff9640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 0x7f605c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.165 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.164+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f60740ac9f0 con 0x7f607c15f470 2024-04-26T22:28:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.164+0000 7f607bfff640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 0x7f605c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.166 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.164+0000 7f607bfff640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 0x7f605c06ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f607c06e830 tx=0x7f6064008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.168 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.168+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f607407ad20 con 0x7f607c15f470 2024-04-26T22:28:50.380 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.380+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp61490", "target": ["mon-mgr", ""]}) -- 0x7f607c0d5300 con 0x7f605c06c980 2024-04-26T22:28:50.386 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:50.387+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:50.386 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:50.387+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:50.386 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:50.387+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:50.387 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:50.387+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:50.387 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-04-26T22:28:50.387+0000 7f9cbb4ea640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:28:50.404 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.404+0000 7f6078ff9640 1 -- 172.21.15.62:0/4166995790 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f607c0d5300 con 0x7f605c06c980 2024-04-26T22:28:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 msgr2=0x7f605c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 0x7f605c06ee40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f607c06e830 tx=0x7f6064008040 comp rx=0 tx=0).stop 2024-04-26T22:28:50.406 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 msgr2=0x7f607c0b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6074009f70 tx=0x7f6074004a40 comp rx=0 tx=0).stop 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 shutdown_connections 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f605c06c980 0x7f605c06ee40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f607c15f470 0x7f607c0b91e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f607c0d1fe0 0x7f607c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 --2- 172.21.15.62:0/4166995790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f607c0d0d30 0x7f607c0bab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 >> 172.21.15.62:0/4166995790 conn(0x7f607c09e510 msgr2=0x7f607c06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 shutdown_connections 2024-04-26T22:28:50.407 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.406+0000 7f6082719640 1 -- 172.21.15.62:0/4166995790 wait complete. 2024-04-26T22:28:50.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol27070 subvol_grp61490 2024-04-26T22:28:50.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.521+0000 7fe7fd220640 1 Processor -- start 2024-04-26T22:28:50.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.521+0000 7fe7fd220640 1 -- start start 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f80d4010 0x7fe7f80d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe7f80d5470 0x7fe7f80d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80b5420 con 0x7fe7f815f5d0 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80d5920 con 0x7fe7f80d5470 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7fd220640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80b50f0 con 0x7fe7f80d4010 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7f6d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7f6d9d640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38334/0 (socket says 172.21.15.62:38334) 2024-04-26T22:28:50.523 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.522+0000 7fe7f6d9d640 1 -- 172.21.15.62:0/66697788 learned_addr learned my addr 172.21.15.62:0/66697788 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f759e640 1 --2- 172.21.15.62:0/66697788 >> v2:172.21.15.174:3300/0 conn(0x7fe7f80d5470 0x7fe7f80d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f659c640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f80d4010 0x7fe7f80d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 -- 172.21.15.62:0/66697788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f80d4010 msgr2=0x7fe7f80d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f80d4010 0x7fe7f80d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 -- 172.21.15.62:0/66697788 >> v2:172.21.15.174:3300/0 conn(0x7fe7f80d5470 msgr2=0x7fe7f80d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/66697788 >> v2:172.21.15.174:3300/0 conn(0x7fe7f80d5470 0x7fe7f80d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.524 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 -- 172.21.15.62:0/66697788 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f80d0a80 con 0x7fe7f815f5d0 2024-04-26T22:28:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.523+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7f808efe0 tx=0x7fe7f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7eb2873dfaf8551 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7f5d9b640 1 -- 172.21.15.62:0/66697788 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0037070 con 0x7fe7f815f5d0 2024-04-26T22:28:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7f5d9b640 1 -- 172.21.15.62:0/66697788 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f0032440 con 0x7fe7f815f5d0 2024-04-26T22:28:50.525 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7f5d9b640 1 -- 172.21.15.62:0/66697788 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0032810 con 0x7fe7f815f5d0 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 -- 172.21.15.62:0/66697788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f815f5d0 msgr2=0x7fe7f80d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7f808efe0 tx=0x7fe7f002a270 comp rx=0 tx=0).stop 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 -- 172.21.15.62:0/66697788 shutdown_connections 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 --2- 172.21.15.62:0/66697788 >> v2:172.21.15.174:3300/0 conn(0x7fe7f80d5470 0x7fe7f80d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f80d4010 0x7fe7f80d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 --2- 172.21.15.62:0/66697788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f815f5d0 0x7fe7f80d3ad0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 -- 172.21.15.62:0/66697788 >> 172.21.15.62:0/66697788 conn(0x7fe7f808ebd0 msgr2=0x7fe7f809e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 -- 172.21.15.62:0/66697788 shutdown_connections 2024-04-26T22:28:50.526 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.524+0000 7fe7fd220640 1 -- 172.21.15.62:0/66697788 wait complete. 2024-04-26T22:28:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 Processor -- start 2024-04-26T22:28:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 -- start start 2024-04-26T22:28:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7f80d4010 0x7fe7f8156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.528 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f815f5d0 0x7fe7f806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f815f9b0 con 0x7fe7f80d5470 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80d1040 con 0x7fe7f80d4010 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7fd220640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80d34f0 con 0x7fe7f815f5d0 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7f659c640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38336/0 (socket says 172.21.15.62:38336) 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.526+0000 7fe7f659c640 1 -- 172.21.15.62:0/3942416412 learned_addr learned my addr 172.21.15.62:0/3942416412 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7f80d4010 0x7fe7f8156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f759e640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f815f5d0 0x7fe7f806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 -- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f815f5d0 msgr2=0x7fe7f806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f815f5d0 0x7fe7f806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 -- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7f80d4010 msgr2=0x7fe7f8156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7f80d4010 0x7fe7f8156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 -- 172.21.15.62:0/3942416412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f80c8f90 con 0x7fe7f80d5470 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7f659c640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00d800 tx=0x7fe7ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.529 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec018020 con 0x7fe7f80d5470 2024-04-26T22:28:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0096a0 con 0x7fe7f80d5470 2024-04-26T22:28:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f80c9270 con 0x7fe7f80d5470 2024-04-26T22:28:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec009a70 con 0x7fe7f80d5470 2024-04-26T22:28:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.527+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f80c9830 con 0x7fe7f80d5470 2024-04-26T22:28:50.530 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.528+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f806a360 con 0x7fe7f80d5470 2024-04-26T22:28:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.529+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fe7ec01b020 con 0x7fe7f80d5470 2024-04-26T22:28:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.529+0000 7fe7df7fe640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 0x7fe7e406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.529+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec08bea0 con 0x7fe7f80d5470 2024-04-26T22:28:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.529+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 0x7fe7e406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.531 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.530+0000 7fe7f6d9d640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 0x7fe7e406ee70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0034000 tx=0x7fe7f002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.533 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.533+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7ec022020 con 0x7fe7f80d5470 2024-04-26T22:28:50.752 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.752+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol27070", "group_name": "subvol_grp61490", "target": ["mon-mgr", ""]}) -- 0x7fe7f80b2f90 con 0x7fe7e406c9b0 2024-04-26T22:28:50.783 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.783+0000 7fe7df7fe640 1 -- 172.21.15.62:0/3942416412 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7f80b2f90 con 0x7fe7e406c9b0 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 msgr2=0x7fe7e406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 0x7fe7e406ee70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0034000 tx=0x7fe7f002e000 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 msgr2=0x7fe7f80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00d800 tx=0x7fe7ec00dcc0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 shutdown_connections 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fe7e406c9b0 0x7fe7e406ee70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7f815f5d0 0x7fe7f806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe7f80d5470 0x7fe7f80ca1e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 --2- 172.21.15.62:0/3942416412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7f80d4010 0x7fe7f8156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 >> 172.21.15.62:0/3942416412 conn(0x7fe7f808ebd0 msgr2=0x7fe7f80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 shutdown_connections 2024-04-26T22:28:50.786 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.785+0000 7fe7fd220640 1 -- 172.21.15.62:0/3942416412 wait complete. 2024-04-26T22:28:50.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol27070 subvol_snap80960 subvol_grp61490 2024-04-26T22:28:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.902+0000 7fa4a32a9640 1 Processor -- start 2024-04-26T22:28:50.903 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.902+0000 7fa4a32a9640 1 -- start start 2024-04-26T22:28:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a32a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa49c1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a32a9640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa49c0d0930 con 0x7fa49c0d5f80 2024-04-26T22:28:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a32a9640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa49c0d0c00 con 0x7fa49c0d5b60 2024-04-26T22:28:50.905 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a32a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa49c0d0ee0 con 0x7fa49c160800 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a1846640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7fa49c0d5f80 0x7fa49c0d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49904/0 (socket says 172.21.15.62:49904) 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.903+0000 7fa4a1846640 1 -- 172.21.15.62:0/286151683 learned_addr learned my addr 172.21.15.62:0/286151683 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa49c1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122709578 0 0) 0x7fa49c0d0930 con 0x7fa49c0d5f80 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa484003520 con 0x7fa49c0d5f80 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3683094805 0 0) 0x7fa484003520 con 0x7fa49c0d5f80 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 msgr2=0x7fa49c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa49c1610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 >> v1:172.21.15.174:6789/0 conn(0x7fa49c0d5b60 legacy=0x7fa49c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c0d11c0 con 0x7fa49c0d5f80 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7fa48c0019f0 con 0x7fa49c0d5f80 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 >> v1:172.21.15.62:6789/0 conn(0x7fa49c0d5f80 legacy=0x7fa49c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.906 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa484003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484004390 0x7fa484004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa4840050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0d0ee0 con 0x7fa4840039e0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa48400b9c0 con 0x7fa484004390 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa48400bbc0 con 0x7fa49c160800 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa484003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a0844640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484004390 0x7fa484004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1846640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa4840050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 -- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 msgr2=0x7fa4840050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c160800 0x7fa4840050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 -- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484004390 msgr2=0x7fa484004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484004390 0x7fa484004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.904+0000 7fa4a1045640 1 -- 172.21.15.62:0/286151683 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa484003520 con 0x7fa4840039e0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.905+0000 7fa4a1045640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa484003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa494009ac0 tx=0x7fa484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab9c692ca5309bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.907 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.905+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494036070 con 0x7fa4840039e0 2024-04-26T22:28:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.905+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa494031400 con 0x7fa4840039e0 2024-04-26T22:28:50.908 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.905+0000 7fa49bfff640 1 -- 172.21.15.62:0/286151683 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494009690 con 0x7fa4840039e0 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 -- 172.21.15.62:0/286151683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 msgr2=0x7fa484003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/286151683 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa484003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa494009ac0 tx=0x7fa484005830 comp rx=0 tx=0).stop 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a1846640 1 -- 172.21.15.62:0/286151683 reap_dead start 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 -- 172.21.15.62:0/286151683 shutdown_connections 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 -- 172.21.15.62:0/286151683 >> 172.21.15.62:0/286151683 conn(0x7fa49c09e550 msgr2=0x7fa49c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 -- 172.21.15.62:0/286151683 shutdown_connections 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.906+0000 7fa4a32a9640 1 -- 172.21.15.62:0/286151683 wait complete. 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.907+0000 7fa4a32a9640 1 Processor -- start 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.907+0000 7fa4a32a9640 1 -- start start 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c0b7e60 0x7fa49c0b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c0ba4b0 0x7fa49c0ba930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.911 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0b6300 con 0x7fa4840039e0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c0d30f0 con 0x7fa49c0b7e60 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a32a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa49c160be0 con 0x7fa49c0ba4b0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38356/0 (socket says 172.21.15.62:38356) 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 -- 172.21.15.62:0/1733012485 learned_addr learned my addr 172.21.15.62:0/1733012485 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a0844640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c0b7e60 0x7fa49c0b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1846640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c0ba4b0 0x7fa49c0ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 -- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c0ba4b0 msgr2=0x7fa49c0ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c0ba4b0 0x7fa49c0ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 -- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c0b7e60 msgr2=0x7fa49c0b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c0b7e60 0x7fa49c0b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 -- 172.21.15.62:0/1733012485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c0bae70 con 0x7fa4840039e0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa4a1045640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa49402abb0 tx=0x7fa494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.908+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494040020 con 0x7fa4840039e0 2024-04-26T22:28:50.912 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.909+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa494003b40 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.909+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494036040 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.909+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c0b8900 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.909+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c0b8ec0 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.910+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49c068550 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.910+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7fa494043020 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.910+0000 7fa499ffb640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 0x7fa48806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.911+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fa4940ab6b0 con 0x7fa4840039e0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.911+0000 7fa4a0844640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 0x7fa48806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:50.913 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.911+0000 7fa4a0844640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 0x7fa48806ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d26a0 tx=0x7fa490009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:50.915 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:50.915+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa49c068550 con 0x7fa4840039e0 2024-04-26T22:28:51.136 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.136+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol27070", "snap_name": "subvol_snap80960", "group_name": "subvol_grp61490", "target": ["mon-mgr", ""]}) -- 0x7fa49c06a360 con 0x7fa48806c980 2024-04-26T22:28:51.151 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.151+0000 7fa499ffb640 1 -- 172.21.15.62:0/1733012485 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa49c06a360 con 0x7fa48806c980 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 msgr2=0x7fa48806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 0x7fa48806ee40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d26a0 tx=0x7fa490009040 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 msgr2=0x7fa49c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa49402abb0 tx=0x7fa494004730 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 shutdown_connections 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7fa48806c980 0x7fa48806ee40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa49c0ba4b0 0x7fa49c0ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa49c0b7e60 0x7fa49c0b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 --2- 172.21.15.62:0/1733012485 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa4840039e0 0x7fa49c15aae0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 >> 172.21.15.62:0/1733012485 conn(0x7fa49c09e550 msgr2=0x7fa49c0d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 shutdown_connections 2024-04-26T22:28:51.154 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.153+0000 7fa4a32a9640 1 -- 172.21.15.62:0/1733012485 wait complete. 2024-04-26T22:28:51.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol27070 subvol_snap80960 key value subvol_grp61490 2024-04-26T22:28:51.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.273+0000 7ff0e3946640 1 Processor -- start 2024-04-26T22:28:51.274 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.274+0000 7ff0e3946640 1 -- start start 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e3946640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff0dc0d1c30 0x7ff0dc0b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e3946640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e3946640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0dc0d3b20 con 0x7ff0dc0d2010 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e3946640 1 -- --> v2:172.21.15.62:3300/0 -- mon_getmap magic: 0 -- 0x7ff0dc0d4380 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e3946640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff0dc0d3dc0 con 0x7ff0dc0d1c30 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 --2- >> v2:172.21.15.62:3300/0 conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38370/0 (socket says 172.21.15.62:38370) 2024-04-26T22:28:51.275 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 -- 172.21.15.62:0/3682028669 learned_addr learned my addr 172.21.15.62:0/3682028669 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e16e2640 1 --2- 172.21.15.62:0/3682028669 >> v2:172.21.15.174:3300/0 conn(0x7ff0dc0d1c30 0x7ff0dc0b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 -- 172.21.15.62:0/3682028669 >> v2:172.21.15.174:3300/0 conn(0x7ff0dc0d1c30 msgr2=0x7ff0dc0b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 --2- 172.21.15.62:0/3682028669 >> v2:172.21.15.174:3300/0 conn(0x7ff0dc0d1c30 0x7ff0dc0b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 -- 172.21.15.62:0/3682028669 >> v1:172.21.15.62:6789/0 conn(0x7ff0dc0d2010 legacy=0x7ff0dc0d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0d3fff640 1 -- 172.21.15.62:0/3682028669 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1375796316 0 0) 0x7ff0dc0d3b20 con 0x7ff0dc0d2010 2024-04-26T22:28:51.276 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.275+0000 7ff0e1ee3640 1 -- 172.21.15.62:0/3682028669 --> v2:172.21.15.62:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc0d40a0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e1ee3640 1 --2- 172.21.15.62:0/3682028669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc08d820 tx=0x7ff0d402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7685c0a934422f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0d3fff640 1 -- 172.21.15.62:0/3682028669 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4036070 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0d3fff640 1 -- 172.21.15.62:0/3682028669 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d4031600 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0d3fff640 1 -- 172.21.15.62:0/3682028669 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40319d0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 -- 172.21.15.62:0/3682028669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 msgr2=0x7ff0dc0d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3682028669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc08d820 tx=0x7ff0d402a730 comp rx=0 tx=0).stop 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 -- 172.21.15.62:0/3682028669 shutdown_connections 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3682028669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0d0710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.277 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3682028669 >> v2:172.21.15.174:3300/0 conn(0x7ff0dc0d1c30 0x7ff0dc0b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.276+0000 7ff0e3946640 1 -- 172.21.15.62:0/3682028669 >> 172.21.15.62:0/3682028669 conn(0x7ff0dc09e560 msgr2=0x7ff0dc0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.277+0000 7ff0e3946640 1 -- 172.21.15.62:0/3682028669 shutdown_connections 2024-04-26T22:28:51.278 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.277+0000 7ff0e3946640 1 -- 172.21.15.62:0/3682028669 wait complete. 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 Processor -- start 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 -- start start 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0dc0d1c30 0x7ff0dc1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 0x7ff0dc1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.278+0000 7ff0e3946640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc0cfea0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e3946640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc0b8620 con 0x7ff0dc0d2010 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e3946640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff0dc0d0c50 con 0x7ff0dc0d1c30 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38380/0 (socket says 172.21.15.62:38380) 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 -- 172.21.15.62:0/3085965187 learned_addr learned my addr 172.21.15.62:0/3085965187 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e1ee3640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 0x7ff0dc1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e0ee1640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0dc0d1c30 0x7ff0dc1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 -- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0dc0d1c30 msgr2=0x7ff0dc1f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.280 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0dc0d1c30 0x7ff0dc1f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 -- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 msgr2=0x7ff0dc1f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 0x7ff0dc1f56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 -- 172.21.15.62:0/3085965187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc0cdc40 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e1ee3640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 0x7ff0dc1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e16e2640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d830 tx=0x7ff0d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800b6c0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0d8019070 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800bac0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.279+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc15f0a0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.281 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.280+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc0cdf20 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.280+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc075a30 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.281+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7ff0d801e460 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.281+0000 7ff0d1ffb640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 0x7ff0c006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.281+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff0d808c1f0 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.281+0000 7ff0e0ee1640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 0x7ff0c006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.282 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.282+0000 7ff0e0ee1640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 0x7ff0c006ee70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc007fd0 tx=0x7ff0cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.285 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.285+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff0dc075a30 con 0x7ff0dc0b63e0 2024-04-26T22:28:51.502 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.502+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol27070", "snap_name": "subvol_snap80960", "key_name": "key", "value": "value", "group_name": "subvol_grp61490", "target": ["mon-mgr", ""]}) -- 0x7ff0dc06a360 con 0x7ff0c006c9b0 2024-04-26T22:28:51.519 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.520+0000 7ff0d1ffb640 1 -- 172.21.15.62:0/3085965187 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff0dc06a360 con 0x7ff0c006c9b0 2024-04-26T22:28:51.521 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 msgr2=0x7ff0c006ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 0x7ff0c006ee70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc007fd0 tx=0x7ff0cc009040 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 msgr2=0x7ff0dc0cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d830 tx=0x7ff0d800dcf0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 shutdown_connections 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7ff0c006c9b0 0x7ff0c006ee70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0dc0d2010 0x7ff0dc1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0dc0d1c30 0x7ff0dc1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 --2- 172.21.15.62:0/3085965187 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0dc0b63e0 0x7ff0dc0cd250 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 >> 172.21.15.62:0/3085965187 conn(0x7ff0dc09e560 msgr2=0x7ff0dc0d5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 shutdown_connections 2024-04-26T22:28:51.522 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.521+0000 7ff0e3946640 1 -- 172.21.15.62:0/3085965187 wait complete. 2024-04-26T22:28:51.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:51.532 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp61490/subvol27070/.meta ./volumes/subvol_grp61490/subvol27070/.meta.stale_snap_section) 2024-04-26T22:28:51.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol27070 subvol_snap80960 subvol_grp61490 2024-04-26T22:28:51.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 Processor -- start 2024-04-26T22:28:51.934 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 -- start start 2024-04-26T22:28:51.936 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5e4c160800 0x7f5e4c1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e4c0d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 -- --> v1:172.21.15.62:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4c0d0930 con 0x7f5e4c0d2150 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c0d2590 con 0x7f5e4c160800 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.934+0000 7f5e52122640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c0d0c00 con 0x7f5e4c0d5b60 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4bfff640 1 --1- >> v1:172.21.15.62:6789/0 conn(0x7f5e4c0d2150 0x7f5e4c0d3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.62:6789/0 says I am v1:172.21.15.62:49930/0 (socket says 172.21.15.62:49930) 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4bfff640 1 -- 172.21.15.62:0/1994626457 learned_addr learned my addr 172.21.15.62:0/1994626457 (peer_addr_for_me v1:172.21.15.62:0/0) 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/1994626457 >> v2:172.21.15.174:3300/0 conn(0x7f5e4c160800 0x7f5e4c1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v1:172.21.15.62:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312498250 0 0) 0x7f5e4c0d0930 con 0x7f5e4c0d2150 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 --> v1:172.21.15.62:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e34003520 con 0x7f5e4c0d2150 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4affd640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e4c0d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v1:172.21.15.62:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1992463080 0 0) 0x7f5e34003520 con 0x7f5e4c0d2150 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 msgr2=0x7f5e4c0d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e4c0d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.937 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 >> v2:172.21.15.174:3300/0 conn(0x7f5e4c160800 msgr2=0x7f5e4c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 --2- 172.21.15.62:0/1994626457 >> v2:172.21.15.174:3300/0 conn(0x7f5e4c160800 0x7f5e4c1610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 --> v1:172.21.15.62:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c0d0ee0 con 0x7f5e4c0d2150 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v1:172.21.15.62:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 523607660 0 0) 0x7f5e440019f0 con 0x7f5e4c0d2150 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 >> v1:172.21.15.62:6789/0 conn(0x7f5e4c0d2150 legacy=0x7f5e4c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e340039b0 0x7f5e34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e34004d40 0x7f5e340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0d2590 con 0x7f5e340039b0 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0d0c00 con 0x7f5e34004d40 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3400b9c0 con 0x7f5e4c0d5b60 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e340039b0 0x7f5e34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4affd640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.935+0000 7f5e4bfff640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e34004d40 0x7f5e340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.938 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 -- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 msgr2=0x7f5e34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0d5b60 0x7f5e34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 -- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e34004d40 msgr2=0x7f5e340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e34004d40 0x7f5e340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 -- 172.21.15.62:0/1994626457 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e34003520 con 0x7f5e340039b0 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e340039b0 0x7f5e34003dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009af0 tx=0x7f5e34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50389b48280e8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c036070 con 0x7f5e340039b0 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c031400 con 0x7f5e340039b0 2024-04-26T22:28:51.939 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.936+0000 7f5e4a7fc640 1 -- 172.21.15.62:0/1994626457 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c009690 con 0x7f5e340039b0 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 -- 172.21.15.62:0/1994626457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e340039b0 msgr2=0x7f5e34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 --2- 172.21.15.62:0/1994626457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e340039b0 0x7f5e34003dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009af0 tx=0x7f5e34005830 comp rx=0 tx=0).stop 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e4bfff640 1 -- 172.21.15.62:0/1994626457 reap_dead start 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 -- 172.21.15.62:0/1994626457 shutdown_connections 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 -- 172.21.15.62:0/1994626457 >> 172.21.15.62:0/1994626457 conn(0x7f5e4c09e550 msgr2=0x7f5e4c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 -- 172.21.15.62:0/1994626457 shutdown_connections 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.937+0000 7f5e52122640 1 -- 172.21.15.62:0/1994626457 wait complete. 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.938+0000 7f5e52122640 1 Processor -- start 2024-04-26T22:28:51.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.938+0000 7f5e52122640 1 -- start start 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e340039b0 0x7f5e4c1560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0b8d00 0x7f5e4c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0d30f0 con 0x7f5e4c0b9180 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0d2e10 con 0x7f5e340039b0 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e52122640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c0b6300 con 0x7f5e4c0b8d00 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e4bfff640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:38402/0 (socket says 172.21.15.62:38402) 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e4bfff640 1 -- 172.21.15.62:0/930733647 learned_addr learned my addr 172.21.15.62:0/930733647 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e340039b0 0x7f5e4c1560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.939+0000 7f5e4affd640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0b8d00 0x7f5e4c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 -- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0b8d00 msgr2=0x7f5e4c1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.943 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0b8d00 0x7f5e4c1572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 -- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e340039b0 msgr2=0x7f5e4c1560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e340039b0 0x7f5e4c1560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 -- 172.21.15.62:0/930733647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c0cd6c0 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e4bfff640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c08fa70 tx=0x7f5e4400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400ec10 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e44018070 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44013420 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c0cd9a0 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.940+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c0cdf60 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.941+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c0b6630 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.941+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 6) ==== 91091+0+0 (secure 0 0 0) 0x7f5e440135c0 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.942+0000 7f5e33fff640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 0x7f5e3806ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.942+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f5e4408bd90 con 0x7f5e4c0b9180 2024-04-26T22:28:51.944 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.942+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 0x7f5e3806ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T22:28:51.945 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.943+0000 7f5e4b7fe640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 0x7f5e3806ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c033000 tx=0x7f5e3c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T22:28:51.946 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:51.946+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mon.0 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5e4c0b6630 con 0x7f5e4c0b9180 2024-04-26T22:28:52.167 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.167+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 --> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol27070", "snap_name": "subvol_snap80960", "group_name": "subvol_grp61490", "target": ["mon-mgr", ""]}) -- 0x7f5e4c06a360 con 0x7f5e3806c980 2024-04-26T22:28:52.203 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.203+0000 7f5e33fff640 1 -- 172.21.15.62:0/930733647 <== mgr.4103 v2:172.21.15.174:6832/1193748239 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e4c06a360 con 0x7f5e3806c980 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 msgr2=0x7f5e3806ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 0x7f5e3806ee40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c033000 tx=0x7f5e3c008030 comp rx=0 tx=0).stop 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 msgr2=0x7f5e4c0ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c08fa70 tx=0x7f5e4400e530 comp rx=0 tx=0).stop 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 shutdown_connections 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:6832/1193748239,v1:172.21.15.174:6833/1193748239] conn(0x7f5e3806c980 0x7f5e3806ee40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5e4c0b9180 0x7f5e4c0ce480 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:52.205 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5e4c0b8d00 0x7f5e4c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 --2- 172.21.15.62:0/930733647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e340039b0 0x7f5e4c1560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T22:28:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 >> 172.21.15.62:0/930733647 conn(0x7f5e4c09e550 msgr2=0x7f5e4c1611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T22:28:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 shutdown_connections 2024-04-26T22:28:52.206 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-26T22:28:52.205+0000 7f5e52122640 1 -- 172.21.15.62:0/930733647 wait complete. 2024-04-26T22:28:52.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T22:28:52.216 DEBUG:teuthology.orchestra.run.smithi062:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp61490/subvol27070/.meta.stale_snap_section ./volumes/subvol_grp61490/subvol27070/.meta) 2024-04-26T22:29:13.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:29:13.963 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:29:44.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:29:44.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:30:14.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:30:14.148 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:30:44.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:30:44.422 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:31:14.572 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:31:14.576 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:31:44.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:31:44.638 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:32:14.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:32:14.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:32:44.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:32:44.931 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:33:14.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:33:14.993 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:33:45.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:33:45.159 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:33:52.220 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 745, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1374, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr raise socket.timeout() socket.timeout 2024-04-26T22:33:52.222 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 697, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1359, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv raise socket.timeout() socket.timeout 2024-04-26T22:34:15.270 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:34:15.273 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:34:45.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:34:45.329 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:35:15.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:35:15.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:35:45.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:35:45.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:36:15.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:36:15.654 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:36:45.746 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:36:45.749 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:37:15.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:37:15.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:37:45.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:37:45.869 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:38:16.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:38:16.014 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:38:46.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:38:46.145 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:39:16.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:39:16.392 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:39:46.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:39:46.514 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:40:16.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:40:16.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:40:46.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:40:46.766 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:41:16.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:41:16.830 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:41:46.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:41:46.886 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:42:17.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:42:17.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:42:47.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:42:47.187 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:43:17.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:43:17.250 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:43:47.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:43:47.372 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:44:17.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:44:17.527 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:44:47.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:44:47.590 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:45:17.684 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:45:17.687 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:45:47.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:45:47.819 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:46:17.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:46:17.892 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:46:47.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:46:47.986 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:47:18.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:47:18.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:47:48.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:47:48.392 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:48:18.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:48:18.524 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:48:48.604 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:48:48.608 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:49:18.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:49:18.734 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:49:48.878 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:49:48.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:50:18.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:50:18.944 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:50:49.072 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:50:49.075 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:51:19.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:51:19.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:51:49.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:51:49.270 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:52:19.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:52:19.390 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:52:49.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:52:49.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:53:19.722 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:53:19.726 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:53:49.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:53:49.857 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:54:19.991 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:54:19.995 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:54:50.071 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:54:50.075 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:55:20.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:55:20.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:55:50.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:55:50.371 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:56:20.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:56:20.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:56:50.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:56:50.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:57:20.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:57:20.713 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:57:50.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:57:50.864 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:58:20.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:58:21.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:58:51.062 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:58:51.066 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:59:21.193 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:59:21.196 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:59:51.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T22:59:51.319 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:00:21.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:00:21.390 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:00:51.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:00:51.521 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:01:21.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:01:21.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:01:51.647 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:01:51.650 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:02:21.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:02:21.794 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:02:51.854 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:02:51.857 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:03:21.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:03:21.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:03:52.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:03:52.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:04:22.204 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:04:22.207 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:04:52.409 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:04:52.412 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:05:22.539 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:05:22.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:05:52.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:05:52.606 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:06:22.740 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:06:22.744 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:06:52.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:06:52.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:07:22.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:07:22.958 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:07:53.082 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:07:53.086 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:08:23.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:08:23.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:08:53.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:08:53.280 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:09:23.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:09:23.418 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:09:53.544 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:09:53.549 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:10:23.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:10:23.618 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:10:53.724 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:10:53.727 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:11:23.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:11:23.853 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:11:53.919 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:11:53.922 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:12:24.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:12:24.032 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:12:54.141 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:12:54.144 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:13:24.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:13:24.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:13:54.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:13:54.538 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:14:24.600 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:14:24.603 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:14:54.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:14:54.691 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:15:24.826 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:15:24.830 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:15:54.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:15:54.892 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:16:25.043 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:16:25.047 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:16:55.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:16:55.201 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:25.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:25.261 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:55.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:17:55.392 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:25.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:25.530 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:55.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:18:55.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:25.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:25.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:55.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:19:55.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:25.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:25.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:56.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:20:56.009 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:26.144 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:26.148 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:56.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:21:56.428 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:22:26.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:22:26.796 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:22:56.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:22:56.870 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:23:27.059 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:23:27.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:23:57.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:23:57.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:24:27.221 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:24:27.224 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:24:57.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:24:57.354 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:25:27.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:25:27.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:25:57.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:25:57.571 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:26:27.717 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:26:27.720 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:26:57.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:26:57.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:27:27.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:27:27.849 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:27:57.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:27:57.946 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:28:28.064 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:28:28.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:28:58.396 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:28:58.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:29:28.528 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:29:28.532 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:29:58.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:29:58.597 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:30:28.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:30:28.735 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:30:58.796 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:30:58.802 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:31:28.861 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:31:28.864 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:31:59.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:31:59.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:32:29.128 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:32:29.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:32:59.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:32:59.199 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:33:29.340 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:33:29.344 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:33:59.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:33:59.412 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:34:29.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:34:29.470 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:34:59.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:34:59.634 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:35:29.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:35:29.773 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:35:59.827 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:35:59.830 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:29.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:36:29.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:00.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:00.089 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:30.307 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:37:30.310 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:00.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:00.364 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:30.442 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:38:30.446 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:00.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:00.529 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:30.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:39:30.669 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:00.728 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:00.731 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:30.850 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:40:30.854 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:00.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:00.917 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:31.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:41:31.115 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:01.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:01.256 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:31.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:42:31.319 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:01.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:01.383 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:31.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:43:31.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:01.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:01.577 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:31.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:44:31.658 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:01.836 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:01.840 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:31.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:45:31.987 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:02.048 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:02.051 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:32.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:46:32.204 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:02.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:02.317 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:32.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:47:32.498 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:02.557 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:02.559 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:32.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:48:32.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:02.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:02.766 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:32.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:49:32.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:02.966 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:02.969 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:33.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:50:33.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:03.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:03.192 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:33.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:51:33.258 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:03.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:03.377 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:33.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:52:33.534 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:03.589 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:03.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:33.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:53:33.744 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:03.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:03.883 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:33.949 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:54:33.953 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:04.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:04.149 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:34.237 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:55:34.241 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:04.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:04.679 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:34.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:56:34.734 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:04.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:04.816 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:34.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:57:34.894 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:05.021 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:05.025 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:35.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:58:35.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:05.206 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:05.210 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:35.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T23:59:35.339 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:00:05.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:00:05.425 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:00:35.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:00:35.551 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:01:05.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:01:05.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:01:35.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:01:35.679 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:02:05.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:02:05.801 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:02:35.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:02:35.938 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:03:06.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:03:06.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:03:36.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:03:36.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:04:06.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:04:06.331 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:04:36.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:04:36.577 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:05:06.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:05:06.707 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:05:36.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:05:36.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:06:06.898 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:06:06.901 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:06:36.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:06:36.963 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:07:07.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:07:07.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:07:37.105 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:07:37.108 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:08:07.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:08:07.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:08:37.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:08:37.244 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:09:07.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:09:07.365 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:09:37.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:09:37.504 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:10:07.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:10:07.590 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:10:37.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:10:37.678 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:11:07.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:11:07.751 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:11:37.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:11:37.815 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:12:07.959 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:12:07.962 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:12:38.098 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:12:38.102 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:13:08.188 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:13:08.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:13:38.325 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:13:38.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:14:08.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:14:08.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:14:38.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:14:38.663 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:15:08.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:15:08.802 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:15:38.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:15:38.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:16:09.076 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:16:09.080 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:16:39.197 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:16:39.201 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:17:09.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:17:09.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:17:39.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:17:39.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:18:09.513 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:18:09.518 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:18:39.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:18:39.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:19:09.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:19:09.719 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:19:39.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:19:39.919 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:20:09.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:20:09.997 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:20:40.139 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:20:40.142 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:21:10.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:21:10.281 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:21:40.608 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:21:40.612 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:22:10.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:22:10.744 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:22:40.843 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:22:40.847 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:23:10.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:23:11.002 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:23:41.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:23:41.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:24:11.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:24:11.195 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:24:41.330 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:24:41.334 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:25:11.469 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:25:11.473 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:25:41.538 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:25:41.542 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:26:11.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:26:11.659 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:26:41.711 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:26:41.716 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:27:11.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:27:11.786 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:27:41.862 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:27:41.869 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:28:12.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:28:12.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:28:42.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:28:42.086 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:29:12.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:29:12.232 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:29:42.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:29:42.371 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:30:12.722 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:30:12.726 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:30:42.802 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:30:42.806 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:31:12.900 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:31:12.904 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:31:43.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:31:43.043 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:32:13.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:32:13.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:32:43.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:32:43.258 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:33:13.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:33:13.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:33:43.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:33:43.545 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:34:13.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:34:13.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:34:43.730 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:34:43.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:35:13.849 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:35:13.853 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:35:43.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:35:43.911 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:36:14.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:36:14.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:36:44.175 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:36:44.179 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:37:14.243 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:37:14.246 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:37:44.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:37:44.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:38:14.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:38:14.513 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:38:44.739 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:38:44.744 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:39:14.828 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:39:14.832 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:39:44.916 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:39:44.920 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:40:15.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:40:15.033 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:40:45.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:40:45.150 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:41:15.224 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:41:15.228 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:41:45.346 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:41:45.350 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:42:15.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:42:15.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:42:45.529 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:42:45.532 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:43:15.652 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:43:15.656 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:43:45.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:43:45.797 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:44:15.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:44:15.866 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:44:46.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:44:46.004 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:45:16.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:45:16.126 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:45:46.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:45:46.194 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:46:16.350 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:46:16.353 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:46:46.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:46:46.484 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:47:16.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:47:16.765 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:47:46.908 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:47:46.911 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:48:16.971 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:48:16.975 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:48:47.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:48:47.088 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:49:17.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:49:17.252 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:49:47.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:49:47.317 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:50:17.444 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:50:17.448 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:50:47.565 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:50:47.568 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:51:17.632 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:51:17.635 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:51:47.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:51:47.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:52:17.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:52:17.960 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:52:48.070 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:52:48.073 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:53:18.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:53:18.162 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:53:48.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:53:48.310 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:54:18.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:54:18.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:54:48.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:54:48.687 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:55:18.755 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:55:18.758 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:55:48.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:55:48.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:56:19.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:56:19.036 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:56:49.090 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:56:49.094 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:57:19.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:57:19.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:57:49.331 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:57:49.335 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:58:19.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:58:19.404 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:58:49.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:58:49.530 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:59:19.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:59:19.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:59:49.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T00:59:49.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:00:19.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:00:19.876 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:00:49.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:00:49.940 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:01:20.008 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:01:20.016 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:01:50.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:01:50.141 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:02:20.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:02:20.315 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:02:50.613 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:02:50.620 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:03:20.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:03:20.754 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:03:50.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:03:50.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:04:21.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:04:21.055 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:04:51.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:04:51.219 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:05:21.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:05:21.282 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:05:51.415 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:05:51.419 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:06:21.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:06:21.558 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:06:51.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:06:51.654 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:07:21.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:07:21.801 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:07:51.878 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:07:51.881 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:08:21.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:08:21.951 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:08:52.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:08:52.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:09:22.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:09:22.171 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:09:52.250 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:09:52.254 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:10:22.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:10:22.348 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:10:52.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:10:52.477 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:11:22.638 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:11:22.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:11:52.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:11:52.762 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:12:22.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:12:22.825 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:12:52.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:12:52.965 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:13:23.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:13:23.102 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:13:53.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:13:53.176 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:14:23.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:14:23.304 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:14:53.389 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:14:53.393 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:15:23.456 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:15:23.459 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:15:53.583 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:15:53.587 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:16:23.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:16:23.716 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:16:53.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:16:53.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:17:23.906 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:17:23.910 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:17:54.028 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:17:54.034 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:18:24.091 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:18:24.095 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:18:54.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:18:54.160 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:19:24.301 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:19:24.305 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:19:54.556 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:19:54.560 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:20:24.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:20:24.740 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:20:54.791 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:20:54.794 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:21:24.863 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:21:24.867 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:21:55.081 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:21:55.084 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:22:25.157 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:22:25.161 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:22:55.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:22:55.305 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:23:25.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:23:25.470 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:23:55.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:23:55.534 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:24:25.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:24:25.655 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:24:55.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:24:55.771 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:25:25.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:25:25.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:25:55.976 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:25:55.979 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:26:26.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:26:26.158 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:26:56.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:26:56.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:27:26.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:27:26.554 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:27:56.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:27:56.632 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:28:26.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:28:26.717 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:28:56.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:28:56.826 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:29:26.892 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:29:26.896 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:29:57.025 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:29:57.029 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:30:27.176 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:30:27.180 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:30:57.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:30:57.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:31:27.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:31:27.362 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:31:57.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:31:57.440 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:32:27.510 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:32:27.514 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:32:57.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:32:57.651 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:33:27.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:33:27.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:33:57.871 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:33:57.875 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:34:27.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:34:27.939 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:34:58.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:34:58.081 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:35:28.140 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:35:28.143 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:35:58.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:35:58.299 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:36:28.417 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:36:28.421 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:36:58.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:36:58.484 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:37:28.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:37:28.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:37:58.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:37:58.894 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:38:28.954 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:38:28.958 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:38:59.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:38:59.130 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:39:29.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:39:29.198 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:39:59.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:39:59.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:40:29.467 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:40:29.471 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:40:59.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:40:59.534 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:41:29.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:41:29.665 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:41:59.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:41:59.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:42:29.922 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:42:29.925 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:43:00.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:43:00.056 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:43:30.174 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:43:30.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:44:00.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:44:00.241 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:44:30.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:44:30.364 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:45:00.473 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:45:00.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:45:30.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:45:30.740 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:46:00.822 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:46:00.826 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:46:30.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:46:30.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:47:01.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:47:01.098 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:47:31.215 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:47:31.219 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:48:01.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:48:01.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:48:31.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:48:31.404 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:49:01.481 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:49:01.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:49:31.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:49:31.551 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:50:01.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:50:01.678 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:50:31.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:50:31.792 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:51:01.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:51:01.863 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:51:31.923 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:51:31.927 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:52:02.045 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:52:02.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:52:32.118 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:52:32.122 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:53:02.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:53:02.193 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:53:32.313 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:53:32.316 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:54:02.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:54:02.379 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:54:32.523 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:54:32.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:55:02.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:55:02.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:55:32.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:55:32.712 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:56:02.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:56:02.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:56:32.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:56:32.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:57:03.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:57:03.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:57:33.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:57:33.167 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:58:03.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:58:03.231 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:58:33.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:58:33.378 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:59:03.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:59:03.512 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:59:33.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T01:59:33.599 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:00:03.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:00:03.736 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:00:33.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:00:33.822 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:01:03.886 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:01:03.889 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:01:34.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:01:34.024 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:02:04.192 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:02:04.195 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:02:34.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:02:34.266 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:03:04.402 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:03:04.405 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:03:34.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:03:34.578 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:04:04.887 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:04:04.891 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:04:34.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:04:34.965 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:05:05.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:05:05.042 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:05:35.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:05:35.232 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:06:05.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:06:05.394 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:06:35.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:06:35.463 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:07:05.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:07:05.579 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:07:35.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:07:35.702 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:08:05.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:08:05.764 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:08:35.883 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:08:35.886 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:09:05.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:09:05.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:09:36.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:09:36.148 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:10:06.290 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:10:06.294 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:10:36.420 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:10:36.423 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:11:06.475 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:11:06.479 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:11:36.613 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:11:36.662 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:12:06.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:12:06.782 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:12:36.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:12:37.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:13:07.064 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:13:07.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:13:37.178 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:13:37.181 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:14:07.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:14:07.270 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:14:37.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:14:37.409 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:15:07.470 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:15:07.474 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:15:37.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:15:37.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:16:07.713 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:16:07.717 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:16:37.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:16:37.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:17:07.923 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:17:07.927 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:17:37.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:17:37.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:18:08.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:18:08.056 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:18:38.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:18:38.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:19:08.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:19:08.322 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:19:38.391 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:19:38.395 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:20:08.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:20:08.508 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:20:38.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:20:38.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:21:08.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:21:08.704 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:21:38.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:21:38.850 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:22:08.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:22:08.937 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:22:39.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:22:39.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:23:09.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:23:09.297 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:23:39.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:23:39.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:24:09.480 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:24:09.483 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:24:39.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:24:39.588 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:25:09.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:25:09.651 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:25:39.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:25:39.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:26:09.792 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:26:09.796 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:26:39.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:26:39.869 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:27:09.930 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:27:09.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:27:40.042 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:27:40.046 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:28:10.116 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:28:10.120 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:28:40.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:28:40.248 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:29:10.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:29:10.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:29:40.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:29:40.471 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:30:10.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:30:10.596 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:30:40.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:30:40.686 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:31:10.743 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:31:10.747 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:31:40.876 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:31:40.879 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:32:10.998 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:32:11.001 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:32:41.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:32:41.247 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:33:11.358 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:33:11.362 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:33:41.421 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:33:41.424 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:34:11.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:34:11.539 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:34:41.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:34:41.686 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:35:11.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:35:11.769 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:35:41.918 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:35:41.922 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:36:12.065 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:36:12.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:36:42.134 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:36:42.138 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:37:12.259 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:37:12.262 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:37:42.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:37:42.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:38:12.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:38:12.464 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:38:42.583 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:38:42.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:39:12.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:39:12.716 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:39:42.794 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:39:42.798 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:40:12.915 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:40:12.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:40:42.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:40:42.980 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:41:13.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:41:13.288 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:41:43.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:41:43.426 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:42:13.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:42:13.497 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:42:43.629 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:42:43.632 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:43:13.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:43:13.695 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:43:43.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:43:43.769 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:44:13.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:44:13.927 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:44:44.056 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:44:44.059 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:45:14.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:45:14.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:45:44.209 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:45:44.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:46:14.308 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:46:14.312 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:46:44.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:46:44.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:47:14.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:47:14.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:47:44.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:47:44.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:48:14.715 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:48:14.719 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:48:44.838 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:48:44.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:49:14.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:49:14.989 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:49:45.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:49:45.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:50:15.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:50:15.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:50:45.438 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:50:45.441 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:51:15.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:51:15.557 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:51:45.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:51:45.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:52:15.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:52:15.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:52:45.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:52:45.850 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:53:16.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:53:16.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:53:46.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:53:46.084 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:54:16.223 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:54:16.227 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:54:46.285 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:54:46.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:55:16.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:55:16.482 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:55:46.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:55:46.537 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:56:16.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:56:16.707 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:56:46.768 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:56:46.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:57:16.883 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:57:16.886 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:57:47.022 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:57:47.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:58:17.261 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:58:17.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:58:47.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:58:47.537 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:59:17.591 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:59:17.595 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:59:47.720 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T02:59:47.724 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:00:17.842 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:00:17.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:00:47.912 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:00:47.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:01:17.987 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:01:17.990 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:01:48.120 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:01:48.124 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:02:18.183 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:02:18.187 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:02:48.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:02:48.301 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:03:18.361 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:03:18.364 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:03:48.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:03:48.437 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:04:18.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:04:18.609 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:04:48.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:04:48.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:05:18.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:05:18.806 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:05:48.942 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:05:48.946 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:06:19.098 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:06:19.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:06:49.303 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:06:49.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:07:19.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:07:19.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:07:49.514 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:07:49.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:08:19.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:08:19.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:08:49.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:08:49.778 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:09:19.839 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:09:19.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:09:49.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:09:49.964 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:10:20.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:10:20.112 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:10:50.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:10:50.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:11:20.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:11:20.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:11:50.443 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:11:50.447 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:12:20.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:12:20.509 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:12:50.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:12:50.632 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:13:20.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:13:20.700 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:13:50.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:13:50.774 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:14:20.853 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:14:20.857 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:14:51.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:14:51.021 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:15:21.082 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:15:21.085 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:15:51.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:15:51.223 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:16:21.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:16:21.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:16:51.496 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:16:51.499 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:17:21.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:17:21.638 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:17:51.697 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:17:51.701 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:18:21.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:18:21.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:18:51.933 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:18:51.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:19:21.996 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:19:22.000 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:19:52.125 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:19:52.129 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:20:22.191 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:20:22.195 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:20:52.262 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:20:52.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:21:22.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:21:22.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:21:52.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:21:52.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:22:22.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:22:22.582 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:22:52.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:22:52.720 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:23:22.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:23:22.875 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:23:52.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:23:52.943 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:24:23.049 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:24:53.170 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:24:53.174 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:25:23.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:25:23.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:25:53.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:25:53.528 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:26:23.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:26:23.625 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:26:53.737 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:26:53.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:27:23.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:27:23.859 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:27:53.935 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:27:53.939 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:28:24.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:28:24.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:28:54.155 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:28:54.158 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:29:24.224 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:29:24.227 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:29:54.349 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:29:54.353 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:30:24.486 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:30:24.490 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:30:54.561 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:30:54.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:31:24.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:31:24.701 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:31:54.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:31:54.768 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:32:24.821 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:32:24.825 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:32:54.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:32:54.959 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:33:25.119 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:33:25.123 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:33:55.191 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:33:55.194 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:34:25.305 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:34:25.308 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:34:55.435 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:34:55.439 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:35:25.649 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:35:25.653 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:35:55.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:35:55.775 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:36:25.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:36:25.848 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:36:55.914 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:36:55.917 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:37:26.047 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:37:26.051 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:37:56.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:37:56.114 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:38:26.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:38:26.261 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:38:56.388 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:38:56.391 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:39:26.451 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:39:26.455 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:39:56.590 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:39:56.594 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:40:26.729 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:40:26.732 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:40:56.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:40:56.813 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:41:26.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:41:26.961 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:41:57.080 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:41:57.083 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:42:27.143 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:42:27.147 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:42:57.273 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:42:57.280 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:43:27.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:43:27.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:43:57.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:43:57.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:44:27.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:44:27.763 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:44:57.814 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:44:57.818 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:45:27.951 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:45:27.955 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:45:58.033 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:45:58.037 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:46:28.096 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:46:28.099 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:46:58.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:46:58.231 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:47:28.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:47:28.543 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:47:58.612 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:47:58.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:48:28.751 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:48:28.755 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:48:58.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:48:58.835 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:49:28.888 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:49:28.891 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:49:59.017 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:49:59.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:50:29.146 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:50:29.150 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:50:59.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:50:59.481 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:51:29.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:51:29.618 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:51:59.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:51:59.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:52:29.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:52:29.827 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:52:59.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:52:59.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:53:30.002 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:53:30.006 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:54:00.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:54:00.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:54:30.280 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:54:30.284 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:55:00.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:55:00.357 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:55:30.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:55:30.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:56:00.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:56:00.573 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:56:30.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:56:30.654 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:57:00.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:57:00.792 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:57:30.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:57:30.932 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:58:00.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:58:00.986 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:58:31.182 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:58:31.186 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:59:01.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:59:01.322 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:59:31.489 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T03:59:31.492 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:00:01.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:00:01.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:00:31.696 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:00:31.700 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:01:01.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:01:01.800 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:01:31.902 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:01:31.905 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:02:01.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:02:01.977 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:02:32.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:02:32.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:03:02.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:03:02.180 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:03:32.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:03:32.247 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:04:02.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:04:02.382 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:04:32.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:04:32.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:05:02.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:05:02.526 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:05:32.644 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:05:32.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:06:02.783 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:06:02.787 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:06:32.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:06:32.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:07:02.944 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:07:02.948 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:07:33.107 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:07:33.111 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:08:03.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:08:03.169 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:08:33.320 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:08:33.324 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:09:03.458 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:09:03.462 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:09:33.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:09:33.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:10:03.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:10:03.724 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:10:33.798 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:10:33.802 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:11:03.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:11:03.928 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:11:34.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:11:34.049 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:12:04.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:12:04.129 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:12:34.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:12:34.210 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:13:04.384 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:13:04.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:13:34.451 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:13:34.455 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:14:04.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:14:04.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:14:34.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:14:34.652 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:15:04.716 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:15:04.720 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:15:34.865 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:15:34.868 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:16:04.986 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:16:04.990 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:16:35.052 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:16:35.055 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:17:05.180 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:17:05.184 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:17:35.327 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:17:35.331 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:18:05.607 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:18:05.611 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:18:35.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:18:35.678 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:19:05.776 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:19:05.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:19:35.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:19:35.885 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:20:06.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:20:06.057 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:20:36.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:20:36.129 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:21:06.207 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:21:06.210 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:21:36.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:21:36.340 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:22:06.401 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:22:06.404 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:22:36.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:22:36.535 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:23:06.628 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:23:06.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:23:36.693 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:23:36.697 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:24:06.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:24:06.809 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:24:36.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:24:36.872 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:25:06.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:25:06.940 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:25:37.054 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:25:37.058 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:26:07.177 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:26:07.180 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:26:37.240 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:26:37.244 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:27:07.330 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:27:07.333 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:27:37.435 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:27:37.438 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:28:07.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:28:07.711 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:28:37.870 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:28:37.873 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:29:07.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:29:07.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:29:38.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:29:38.043 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:30:08.162 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:30:08.166 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:30:38.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:30:38.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:31:08.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:31:08.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:31:38.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:31:38.449 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:32:08.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:32:08.519 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:32:38.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:32:38.738 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:33:08.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:33:08.881 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:33:38.944 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:33:38.947 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:34:09.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:34:09.059 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:34:39.212 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:34:39.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:35:09.282 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:35:09.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:35:39.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:35:39.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:36:09.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:36:09.522 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:36:39.731 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:36:39.735 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:37:09.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:37:09.824 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:37:39.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:37:39.895 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:38:10.073 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:38:10.076 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:38:40.220 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:38:40.224 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:39:10.293 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:39:10.296 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:39:40.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:39:40.409 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:40:10.470 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:40:10.473 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:40:40.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:40:40.546 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:41:10.672 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:41:10.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:41:40.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:41:40.749 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:42:10.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:42:10.819 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:42:40.948 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:42:40.951 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:43:11.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:43:11.020 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:43:41.083 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:43:41.086 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:44:11.189 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:44:11.192 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:44:41.310 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:44:41.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:45:11.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:45:11.386 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:45:41.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:45:41.516 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:46:11.590 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:46:11.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:46:41.823 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:46:41.827 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:47:11.969 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:47:11.973 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:47:42.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:47:42.033 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:48:12.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:48:12.166 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:48:42.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:48:42.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:49:12.372 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:49:12.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:49:42.472 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:49:42.475 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:50:12.692 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:50:12.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:50:42.757 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:50:42.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:51:12.879 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:51:12.883 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:51:42.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:51:42.980 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:52:13.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:52:13.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:52:43.187 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:52:43.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:53:13.326 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:53:13.330 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:53:43.411 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:53:43.414 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:54:13.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:54:13.503 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:54:43.563 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:54:43.566 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:55:13.813 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:55:13.817 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:55:43.936 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:55:43.940 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:56:14.016 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:56:14.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:56:44.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:56:44.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:57:14.276 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:57:14.279 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:57:44.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:57:44.346 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:58:14.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:58:14.466 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:58:44.602 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:58:44.605 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:59:14.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:59:14.662 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:59:44.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T04:59:44.782 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:00:14.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:00:14.920 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:00:45.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:00:45.057 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:01:15.187 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:01:15.190 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:01:45.342 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:01:45.345 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:02:15.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:02:15.410 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:02:45.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:02:45.481 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:03:15.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:03:15.628 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:03:45.681 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:03:45.684 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:04:15.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:04:15.814 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:04:45.950 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:04:45.953 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:05:16.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:05:16.050 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:05:46.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:05:46.222 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:06:16.286 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:06:16.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:06:46.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:06:46.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:07:16.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:07:16.496 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:07:46.564 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:07:46.569 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:08:16.694 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:08:16.697 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:08:46.859 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:08:46.862 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:09:16.913 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:09:16.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:09:47.027 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:09:47.031 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:10:17.174 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:10:17.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:10:47.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:10:47.250 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:11:17.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:11:17.380 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:11:47.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:11:47.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:12:17.579 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:12:17.583 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:12:47.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:12:47.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:13:17.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:13:17.835 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:13:48.053 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:13:48.057 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:14:18.124 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:14:18.128 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:14:48.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:14:48.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:15:18.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:15:18.323 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:15:48.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:15:48.454 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:16:18.513 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:16:18.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:16:48.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:16:48.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:17:18.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:17:18.778 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:17:48.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:17:48.828 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:18:18.960 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:18:18.963 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:18:49.027 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:18:49.031 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:19:19.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:19:19.103 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:19:49.168 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:19:49.171 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:20:19.306 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:20:19.310 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:20:49.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:20:49.372 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:21:19.508 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:21:19.514 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:21:49.639 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:21:49.680 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:22:19.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:22:19.751 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:22:49.875 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:22:49.879 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:23:19.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:23:19.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:23:50.236 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:23:50.240 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:24:20.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:24:20.338 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:24:50.400 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:24:50.403 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:25:20.537 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:25:20.540 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:25:50.633 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:25:50.636 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:26:20.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:26:20.704 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:26:50.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:26:50.847 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:27:20.980 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:27:20.983 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:27:51.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:27:51.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:28:21.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:28:21.318 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:28:51.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:28:51.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:29:21.520 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:29:21.524 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:29:51.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:29:51.667 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:30:21.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:30:21.823 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:30:52.184 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:30:52.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:31:22.314 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:31:22.318 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:31:52.383 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:31:52.387 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:32:22.516 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:32:22.519 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:32:52.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:32:52.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:33:22.700 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:33:22.704 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:33:52.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:33:52.808 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:34:22.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:34:22.938 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:34:52.992 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:34:52.995 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:35:23.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:35:23.139 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:35:53.281 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:35:53.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:36:23.352 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:36:23.356 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:36:53.466 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:36:53.470 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:37:23.587 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:37:23.591 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:37:53.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:37:53.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:38:23.773 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:38:23.776 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:38:53.911 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:38:53.915 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:39:24.234 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:39:24.237 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:39:54.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:39:54.316 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:40:24.419 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:40:24.422 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:40:54.549 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:40:54.552 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:41:24.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:41:24.690 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:41:54.766 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:41:54.770 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:42:24.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:42:24.909 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:42:55.078 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:42:55.084 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:43:25.149 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:43:25.153 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:43:55.296 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:43:55.299 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:44:25.434 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:44:25.438 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:44:55.505 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:44:55.508 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:45:25.595 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:45:25.598 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:45:55.775 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:45:55.778 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:46:26.147 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:46:26.151 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:46:56.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:46:56.231 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:47:26.315 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:47:26.318 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:47:56.453 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:47:56.456 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:48:26.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:48:26.578 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:48:56.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:48:56.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:49:26.769 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:49:26.773 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:49:56.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:49:56.828 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:50:26.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:50:26.897 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:50:57.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:50:57.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:51:27.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:51:27.152 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:51:57.228 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:51:57.232 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:52:27.359 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:52:27.362 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:52:57.447 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:52:57.450 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:53:27.512 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:53:27.515 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:53:57.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:53:57.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:54:27.762 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:54:27.766 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:54:57.833 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:54:57.837 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:55:28.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:55:28.014 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:55:58.127 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:55:58.130 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:56:28.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:56:28.277 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:56:58.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:56:58.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:57:28.408 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:57:28.411 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:57:58.547 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:57:58.551 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:58:28.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:58:28.707 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:58:58.767 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:58:58.770 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:59:28.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:59:28.901 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:59:59.010 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T05:59:59.014 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:00:29.082 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:00:29.085 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:00:59.213 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:00:59.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:01:29.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:01:29.463 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:01:59.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:01:59.535 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:02:29.669 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:02:29.673 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:02:59.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:02:59.736 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:03:29.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:03:29.808 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:03:59.919 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:03:59.922 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:04:30.084 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:04:30.087 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:05:00.379 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:05:00.382 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:05:30.454 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:05:30.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:06:00.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:06:00.522 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:06:30.645 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:06:30.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:07:00.770 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:07:00.773 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:07:30.833 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:07:30.837 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:08:00.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:08:00.959 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:08:31.095 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:08:31.098 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:09:01.153 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:09:01.156 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:09:31.281 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:09:31.284 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:10:01.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:10:01.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:10:31.468 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:10:31.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:11:01.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:11:01.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:11:31.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:11:31.749 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:12:01.810 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:12:01.814 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:12:31.965 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:12:31.969 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:13:02.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:13:02.040 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:13:32.377 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:13:32.395 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:14:02.482 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:14:02.485 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:14:32.584 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:14:32.588 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:15:02.709 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:15:02.713 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:15:32.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:15:32.835 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:16:02.904 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:16:02.908 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:16:33.034 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:16:33.038 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:17:03.181 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:17:03.185 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:17:33.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:17:33.253 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:18:03.376 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:18:03.379 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:18:33.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:18:33.501 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:19:03.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:19:03.681 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:19:33.808 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:19:33.811 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:20:03.956 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:20:03.959 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:20:34.203 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:20:34.206 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:21:04.284 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:21:04.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:21:34.355 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:21:34.359 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:22:04.494 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:22:04.497 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:22:34.575 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:22:34.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:23:04.688 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:23:04.692 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:23:34.794 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:23:34.798 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:24:04.932 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:24:04.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:24:35.005 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:24:35.009 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:25:05.159 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:25:05.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:25:35.299 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:25:35.303 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:26:05.357 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:26:05.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:26:35.493 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:26:35.496 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:27:05.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:27:05.559 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:27:35.625 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:27:35.629 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:28:05.759 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:28:05.763 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:28:35.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:28:35.876 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:29:06.195 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:29:06.199 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:29:36.277 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:29:36.280 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:30:06.405 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:30:06.409 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:30:36.588 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:30:36.591 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:31:06.675 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:31:06.678 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:31:36.756 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:31:36.760 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:32:06.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:32:06.900 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:32:36.957 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:32:36.962 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:33:07.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:33:07.033 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:33:37.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:33:37.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:34:07.248 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:34:07.252 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:34:37.319 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:34:37.322 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:35:07.476 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:35:07.479 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:35:37.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:35:37.618 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:36:07.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:36:07.690 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:36:37.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:36:37.764 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:37:07.897 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:37:07.901 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:37:37.977 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:37:37.981 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:38:08.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:38:08.074 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:38:38.246 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:38:38.250 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:39:08.386 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:39:08.389 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:39:38.507 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:39:38.511 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:40:08.570 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:40:08.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:40:38.701 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:40:38.705 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:41:08.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:41:08.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:41:38.921 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:41:40.492 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:42:10.635 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:42:10.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:42:40.726 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:42:40.730 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:43:10.846 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:43:10.849 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:43:40.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:43:40.921 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:44:10.990 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:44:10.994 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:44:41.138 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:44:41.177 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:45:11.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:45:11.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:45:41.665 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:45:41.669 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:46:11.830 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:46:11.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:46:41.968 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:46:41.971 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:47:12.099 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:47:12.103 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:47:42.171 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:47:42.174 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:48:12.317 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:48:12.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:48:42.430 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:48:42.434 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:49:12.545 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:49:12.549 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:49:42.650 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:49:42.653 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:50:12.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:50:12.784 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:50:42.877 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:50:42.884 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:51:13.024 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:51:13.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:51:43.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:51:43.160 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:52:13.235 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:52:13.238 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:52:43.316 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:52:43.320 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:53:13.437 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:53:13.440 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:53:43.568 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:53:43.572 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:54:13.664 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:54:13.668 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:54:43.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:54:43.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:55:13.909 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:55:13.913 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:55:44.030 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:55:44.034 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:56:14.160 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:56:14.164 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:56:44.266 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:56:44.270 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:57:14.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:57:14.433 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:57:44.552 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:57:44.556 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:58:14.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:58:14.702 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:58:44.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:58:44.841 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:59:14.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:59:14.970 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:59:45.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T06:59:45.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:00:15.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:00:15.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:00:45.256 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:00:45.259 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:01:15.380 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:01:15.384 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:01:45.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:01:45.522 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:02:15.657 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:02:15.684 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:02:45.779 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:02:45.782 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:15.926 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:15.930 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:46.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:03:46.070 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:16.213 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:16.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:46.288 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:04:46.292 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:16.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:16.426 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:46.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:05:46.557 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:16.666 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:16.669 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:46.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:06:46.792 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:07:16.927 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:07:16.930 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:07:46.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:07:46.990 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:08:17.113 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:08:17.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:08:47.185 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:08:47.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:09:17.257 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:09:17.261 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:09:47.370 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:09:47.374 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:10:17.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:10:17.503 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:10:47.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:10:47.590 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:11:17.660 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:11:17.663 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:11:47.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:11:47.735 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:12:17.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:12:17.791 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:12:47.860 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:12:47.863 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:13:17.923 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:13:17.927 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:13:48.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:13:48.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:14:18.075 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:14:18.079 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:14:48.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:14:48.134 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:15:18.202 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:15:18.206 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:15:48.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:15:48.277 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:16:18.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:16:18.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:16:48.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:16:48.529 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:17:18.582 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:17:18.586 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:17:48.721 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:17:48.725 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:18:18.869 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:18:18.872 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:18:48.940 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:18:48.943 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:19:19.088 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:19:19.092 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:19:49.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:19:49.155 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:20:19.256 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:20:19.260 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:20:49.403 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:20:49.407 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:21:19.461 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:21:19.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:21:49.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:21:49.609 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:22:19.732 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:22:19.736 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:22:49.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:22:49.819 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:23:19.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:23:19.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:23:50.109 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:23:50.113 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:24:20.205 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:24:20.208 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:24:50.318 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:24:50.322 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:25:20.382 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:25:20.386 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:25:50.521 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:25:50.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:26:20.643 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:26:20.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:26:50.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:26:50.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:27:20.928 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:27:20.931 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:27:51.050 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:27:51.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:28:21.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:28:21.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:28:51.221 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:28:51.225 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:29:21.302 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:29:21.305 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:29:51.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:29:51.379 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:30:21.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:30:21.437 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:30:51.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:30:51.507 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:31:21.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:31:21.589 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:31:51.725 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:31:51.729 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:32:21.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:32:21.816 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:32:51.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:32:51.913 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:33:22.040 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:33:22.044 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:33:52.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:33:52.149 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:34:22.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:34:22.296 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:34:52.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:34:52.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:35:22.527 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:35:22.530 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:35:52.611 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:35:52.616 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:36:22.747 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:36:22.750 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:36:52.807 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:36:52.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:37:22.891 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:37:22.895 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:37:53.029 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:37:53.032 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:38:23.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:38:23.105 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:38:53.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:38:53.253 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:39:23.404 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:39:23.407 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:39:53.551 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:39:53.554 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:40:23.614 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:40:23.617 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:40:53.753 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:40:53.757 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:41:23.816 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:41:23.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:41:53.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:41:53.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:42:24.077 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:42:24.080 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:42:54.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:42:54.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:43:24.312 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:43:24.316 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:43:54.375 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:43:54.379 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:44:24.530 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:44:24.534 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:44:54.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:44:54.689 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:45:24.815 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:45:24.818 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:45:54.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:45:54.878 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:46:25.008 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:46:25.012 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:46:55.173 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:46:55.176 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:47:25.320 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:47:25.626 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:47:55.707 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:47:55.710 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:48:25.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:48:25.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:48:55.984 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:48:55.988 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:26.123 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:26.127 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:56.211 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:49:56.214 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:26.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:26.278 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:56.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:50:56.416 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:26.518 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:26.521 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:56.640 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:51:56.643 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:26.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:26.740 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:56.874 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:52:56.878 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:27.013 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:27.016 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:57.101 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:53:57.105 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:27.249 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:27.252 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:57.378 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:54:57.383 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:55:27.500 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:55:27.503 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:55:57.580 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:55:57.584 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:56:27.719 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:56:27.723 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:56:57.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:56:57.784 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:57:27.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:57:27.932 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:57:57.994 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:57:57.998 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:58:28.057 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:58:28.061 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:58:58.130 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:58:58.134 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:59:28.274 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:59:28.278 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:59:58.429 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T07:59:58.433 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:00:28.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:00:28.563 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:00:58.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:00:58.618 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:01:28.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:01:28.749 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:01:58.817 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:01:58.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:02:28.947 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:02:28.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:02:59.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:02:59.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:03:29.232 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:03:29.236 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:03:59.387 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:03:59.390 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:04:29.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:04:29.513 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:04:59.656 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:04:59.660 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:05:29.736 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:05:29.740 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:05:59.804 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:05:59.808 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:06:29.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:06:29.896 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:07:00.035 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:07:00.038 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:07:30.156 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:07:30.159 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:08:00.287 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:08:00.290 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:08:30.418 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:08:30.421 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:09:00.531 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:09:00.535 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:09:30.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:09:30.681 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:10:00.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:10:00.737 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:10:30.805 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:10:30.808 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:11:00.910 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:11:00.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:11:31.066 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:11:31.070 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:12:01.222 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:12:01.226 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:12:31.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:12:31.302 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:13:01.432 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:13:01.435 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:13:31.553 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:13:31.557 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:14:01.683 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:14:01.686 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:14:31.997 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:14:32.000 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:15:02.126 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:15:02.130 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:15:32.264 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:15:32.267 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:16:02.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:16:02.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:16:32.426 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:16:32.430 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:17:02.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:17:02.502 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:17:32.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:17:32.563 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:18:02.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:18:02.630 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:18:32.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:18:32.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:19:02.784 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:19:02.788 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:19:32.844 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:19:32.847 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:20:02.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:20:02.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:20:33.108 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:20:33.111 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:21:03.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:21:03.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:21:33.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:21:33.339 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:22:03.424 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:22:03.428 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:22:33.571 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:22:33.575 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:23:03.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:23:03.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:23:33.856 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:23:33.860 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:24:03.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:24:03.941 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:24:34.059 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:24:34.062 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:25:04.198 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:25:04.202 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:25:34.335 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:25:34.338 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:26:04.407 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:26:04.411 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:26:34.562 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:26:34.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:27:04.642 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:27:04.646 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:27:34.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:27:34.768 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:28:04.852 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:28:04.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:28:34.983 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:28:34.987 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:29:05.114 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:29:05.118 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:29:35.252 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:29:35.256 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:30:05.348 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:30:05.352 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:30:35.487 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:30:35.491 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:31:05.592 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:31:05.595 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:31:35.655 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:31:35.659 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:32:05.778 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:32:05.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:32:35.907 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:32:35.914 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:33:05.970 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:33:05.974 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:33:36.110 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:33:36.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:34:06.232 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:34:06.235 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:34:36.338 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:34:36.341 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:35:06.450 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:35:06.453 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:35:36.522 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:35:36.525 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:36:06.610 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:36:06.614 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:36:36.673 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:36:36.677 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:37:06.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:37:06.745 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:37:36.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:37:36.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:38:06.973 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:38:06.977 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:38:37.087 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:38:37.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:39:07.154 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:39:07.157 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:39:37.231 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:39:37.235 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:40:07.369 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:40:07.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:40:37.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:40:37.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:41:07.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:41:07.553 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:41:37.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:41:37.609 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:42:07.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:42:08.208 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:42:38.283 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:42:38.287 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:43:08.413 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:43:08.416 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:43:38.535 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:43:38.538 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:44:08.674 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:44:08.677 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:44:38.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:44:38.792 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:45:08.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:45:08.938 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:45:39.090 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:45:39.094 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:46:09.196 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:46:09.200 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:46:39.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:46:39.344 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:47:09.463 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:47:09.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:47:39.526 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:47:39.530 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:48:09.601 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:48:09.605 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:48:39.661 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:48:39.665 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:49:09.733 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:49:09.736 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:49:39.797 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:49:39.800 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:50:09.868 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:50:09.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:50:39.924 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:50:39.927 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:51:09.989 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:51:09.993 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:51:40.051 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:51:40.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:52:10.106 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:52:10.110 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:52:40.165 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:52:40.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:53:10.230 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:53:10.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:53:40.298 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:53:40.302 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:54:10.366 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:54:10.373 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:54:40.433 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:54:40.437 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:55:10.503 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:55:10.506 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:55:40.560 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:55:40.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:56:10.685 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:56:10.689 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:56:40.812 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:56:40.815 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:57:10.917 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:57:10.921 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:57:41.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:57:41.050 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:58:11.210 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:58:11.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:58:41.341 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:58:41.345 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:59:11.471 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:59:11.474 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:59:41.524 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T08:59:41.527 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:00:11.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:00:11.662 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:00:41.803 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:00:41.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:01:11.893 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:01:11.897 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:01:42.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:01:42.026 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:02:12.169 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:02:12.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:02:42.258 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:02:42.262 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:03:12.371 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:03:12.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:03:42.644 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:03:42.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:04:12.723 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:04:12.728 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:04:42.811 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:04:42.815 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:05:12.951 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:05:12.955 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:05:43.020 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:05:43.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:06:13.112 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:06:13.115 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:06:43.232 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:06:43.236 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:07:13.322 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:07:13.325 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:07:43.460 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:07:43.464 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:08:13.525 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:08:13.529 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:08:43.621 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:08:43.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:09:13.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:09:13.745 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:09:43.889 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:09:43.893 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:10:13.964 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:10:13.968 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:10:44.019 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:10:44.022 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:11:14.085 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:11:14.088 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:11:44.151 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:11:44.154 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:12:14.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:12:14.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:12:44.278 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:12:44.283 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:13:14.334 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:13:14.338 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:13:44.464 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:13:44.468 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:14:14.593 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:14:14.598 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:14:44.658 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:14:44.662 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:15:14.780 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:15:14.783 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:15:44.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:15:44.844 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:16:14.890 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:16:14.894 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:16:44.962 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:16:44.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:17:15.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:17:15.104 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:17:45.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:17:45.250 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:18:15.394 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:18:15.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:18:45.517 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:18:45.521 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:19:15.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:19:15.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:19:45.734 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:19:45.737 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:20:15.818 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:20:15.822 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:20:45.929 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:20:45.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:21:16.067 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:21:16.071 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:21:46.164 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:21:46.167 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:22:16.244 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:22:16.247 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:22:46.324 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:22:46.328 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:23:16.397 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:23:16.401 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:23:46.499 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:23:46.503 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:24:16.567 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:24:16.570 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:24:46.622 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:24:46.626 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:25:16.686 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:25:16.690 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:25:46.820 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:25:46.824 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:26:16.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:26:16.966 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:26:47.015 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:26:47.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:27:17.129 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:27:17.133 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:27:47.184 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:27:47.187 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:28:17.247 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:28:17.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:28:47.395 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:28:47.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:29:17.474 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:29:17.478 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:29:47.533 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:29:47.536 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:30:17.605 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:30:17.608 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:30:47.667 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:30:47.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:31:17.741 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:31:17.745 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:31:47.880 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:31:47.886 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:32:17.967 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:32:17.970 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:32:48.105 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:32:48.108 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:33:18.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:33:18.231 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:33:48.300 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:33:48.304 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:34:18.446 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:34:18.450 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:34:48.559 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:34:48.943 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:35:19.023 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:35:19.027 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:35:49.086 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:35:49.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:36:19.208 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:36:19.212 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:36:49.264 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:36:49.268 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:37:19.336 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:37:19.340 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:37:49.399 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:37:49.403 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:38:19.492 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:38:19.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:38:49.554 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:38:49.558 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:39:19.627 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:39:19.630 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:39:49.687 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:39:49.691 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:40:19.760 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:40:19.764 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:40:49.831 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:40:49.834 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:41:19.903 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:41:19.907 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:41:49.965 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:41:49.968 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:42:20.055 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:42:20.058 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:42:50.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:42:50.115 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:43:20.184 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:43:20.187 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:43:50.245 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:43:50.249 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:44:20.328 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:44:20.332 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:44:50.390 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:44:50.393 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:45:20.449 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:45:20.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:45:50.508 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:45:50.511 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:46:20.577 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:46:20.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:46:50.634 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:46:50.638 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:47:20.690 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:47:20.693 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:47:50.754 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:47:50.758 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:48:20.819 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:48:20.823 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:48:50.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:48:50.884 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:49:20.946 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:49:20.949 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:49:51.001 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:49:51.005 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:50:21.139 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:50:21.142 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:50:51.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:50:51.222 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:51:21.294 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:51:21.297 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:51:51.351 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:51:51.354 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:52:21.423 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:52:21.426 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:52:51.478 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:52:51.481 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:53:21.550 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:53:21.555 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:53:51.620 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:53:51.623 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:54:21.679 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:54:21.682 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:54:51.761 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:54:51.765 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:55:21.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:55:21.827 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:55:51.882 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:55:51.885 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:56:21.937 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:56:21.941 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:56:52.009 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:56:52.012 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:57:22.082 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:57:22.085 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:57:52.161 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:57:52.164 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:58:22.227 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:58:22.231 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:58:52.292 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:58:52.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:59:22.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:59:22.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:59:52.506 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T09:59:52.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:00:22.704 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:00:22.708 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:00:52.785 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:00:52.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:01:22.892 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:01:22.895 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:01:52.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:01:52.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:02:23.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:02:23.041 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:02:53.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:02:53.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:03:23.360 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:03:23.363 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:03:53.491 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:03:53.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:04:23.630 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:04:23.635 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:04:53.752 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:04:53.755 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:05:23.840 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:05:23.843 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:05:53.978 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:05:53.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:06:24.100 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:06:24.104 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:06:54.214 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:06:54.217 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:07:24.344 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:07:24.347 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:07:54.519 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:07:54.523 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:08:24.646 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:08:24.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:08:54.706 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:08:54.709 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:09:24.781 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:09:24.785 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:09:54.845 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:09:54.848 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:10:24.925 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:10:24.928 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:10:55.097 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:10:55.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:11:25.219 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:11:25.222 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:11:55.365 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:11:55.369 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:12:25.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:12:25.483 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:12:55.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:12:55.612 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:13:25.698 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:13:25.702 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:13:55.837 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:13:55.840 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:14:25.901 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:14:25.903 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:14:55.955 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:14:55.959 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:15:26.036 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:15:26.039 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:15:56.166 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:15:56.170 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:16:26.255 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:16:26.259 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:16:56.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:16:56.371 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:17:26.498 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:17:26.502 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:17:56.636 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:17:56.639 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:18:26.708 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:18:26.712 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:18:56.777 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:18:56.780 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:19:26.849 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:19:26.852 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:19:56.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:19:56.908 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:20:26.979 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:20:26.983 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:20:57.145 DEBUG:teuthology.orchestra.run.smithi062:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:20:57.148 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-27T10:21:22.705 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-27T10:21:22.706 DEBUG:teuthology.task.console_log:Killing console logger for smithi062 2024-04-27T10:21:22.706 DEBUG:teuthology.task.console_log:Killing console logger for smithi174 2024-04-27T10:21:22.706 DEBUG:teuthology.exit:Finished running handlers